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DAHC- 1S-H9-C-02R5,  under  the  direction  of  L.  Kleinrock,  Principal  Investi¬ 
gator,  and  H.  Estrin,  M.  Melkanoff,  and  R.  Muntz,  Co-Principal  Investigators, 
in  the  Computer  Science  Denartment  of  the  School  of  Engineering  and  Applied 
Science,  University  of  California,  Los  Angeles, 

This  report  was  the  basis  of  a  Ph.D,  dissertation  f.Iune  1971)  sub¬ 
mitted  by  the  author  under  the  chairmanship  of  Leonard  Kleinrock. 
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ABSTRACT 


The  ARPA  (Advanced  Research  Projects  Agency)  conputer  network  involves 
the  interconnection  of  about  twenty  (as  of  1971)  different  research  computers 
across  the  country  by  means  of  a  store-and-forward  message  switching  net.  The 
development  of  such  a  network  is  an  expensive  and  complicated  undertaking 
which  involves  a  variety  of  engineering  trade-offs  and  decisions.  Since 
there  has  been  little  prior  experience  which  directly  relates  to  the  design 
of  such  a  network,  an  extensive  measurement  and  evaluation  capability  was 
included  in  the  message  switching  computers  (the  Interface  Message  Processors 
or  IMF’s).  UCLA  was  designated  to  be  the  Network  Measurement  Center  with  the 
responsibility  of  defining  the  measurements  that  were  necessary  for  the 
support  of  the  analytic  and  simulation  model  activities,  and  to  determine  the 
performance  of  the  network  by  the  use  of  these  measurement  facilities.  The 
primary  concern  of  this  report  has  been  with  the  development  of  such  a 
measurement  capability  and  the  utilization  of  this  capability  to  create  (and 
iteratively  improve)  analytic  models  of  the  network  behavior  as  well  as  the 
true  system  parameters. 

The  measurement  facilities  which  were  designed  into  the  network  included: 
accumulated  data  such  as  histograms  and  totals;  snap-shot  data  relating  to 
queue  lengths  and  routing  Information;  and  traces  of  message  flow  through  the 
network.  Any  of  these  measurement  routines  can  be  selectively  enabled  at  one 
or  more  of  the  network  IMF's  to  avoid  an  excessive  data  collection  and  artifact 
problem.  In  addition  to  the  selective  control  over  the  measurements,  artifact 
was  further  reduced  by  the  careful  selection  of  the  variables  to  be  measured, 
and  by  the  development  of  measurement  techniques  such  as  the  use  of  non-uniform 
(logarithmic)  scale  histograms  for  data  which  was  expected  to  have  an  exponential¬ 
like  distribution. 
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An  extensive  artificial  traffic  generation  capability  was  also 
developed  at  UCLA  and  was  utilized  with  the  measurement  facilities  to  verify 
and  improve  analytic  models  of  the  system  behavior.  In  some  instances,  these 
models  were  initially  developed  based  on  a  priori  expectations  of  the  system 
behavior,  while  in  other  cases,  the  models  were  devised  from  observed  system 
performance  data.  In  both  situations,  the  iterative  usage  of  model-building 
and  experimental  verification  was  found  to  provide  valuable  feedback  regarding 
needed  improvements  in  the  models,  and  resulted  in  good  agreement  between  the 
final  models  and  the  observed  system  behavior. 

The  models  developed  were  chosen  tc  represent  a  reasonably  broad  spectrum 
of  interest,  and  included  priority  traffic  analysis,  message  segmentation, 
buffering  consideration,  message  delay  considerations,  and  network  thru-put. 

The  models  consisted  of  both  original  contributions,  and  extensions  to  previous 
analytic  results,  although  the  most  significant  aspect  of  the  research  is  felt 
to  be  the  demonstration  that  an  extensive  set  of  measurement  facilities  can  be 
developed  utiJ '.zing  the  existing  network  resources,  and  that  such  measurements 
can  provide  valuable  insight  into  the  network  behavior  which  can  be  utilized 
as  feedback  in  an  iterative  model-building,  evaluation  and  design  effort. 
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INTRODUCTION 

In  the  quarter-century  since  the  first  electronic  digital 
aornputer  was  developed,  the  computer  industry  has  grown  at  a  rapid  rate 
with  hundreds  of  different  computers  being  marketed  and  scores  of  pro¬ 
gramming  languages  being  written.  This  relatively  unrestricted  prolif¬ 
eration  of  hardware  and  software  systems  has  contributed  to  the  rapid 
development  of  computer  technology,  but  has  also  created  a  tremendous 
inccmpatability  problem.  As  a  consequence,  many  programs  have  been 
rewritten  at  each  instance  in  which  they  were  to  be  utilized  on  hard¬ 
ware  unlike  that  of  the  originating  facility,  while  in  other  cases, 
programs  have  been  so  dependent  on  special  hardware  or  operating  system 
characteristics  that  they  have  not  been  exportable  in  any  practical 
manner. 

Standardization  of  hardware  and  software  systems  has  been 
advocated  as  a  solution  to  these  incanpatibility  problems,  but  has  not 
been  viable  in  the  past,  nor  does  it  appear  to  be  in  the  near  future. 
Same  progress  has  been  made  in  the  areas  of  ASCII  character  code  stand¬ 
ardization,  data  carmuni cation  interface  standards,  3rd  seme  language 
definitions,  but  acceptable  industry-wide  standards  ozn  critical  factors 
such  as  data  structures  and  access  methods  have  not  been  established. 
Indeed,  not  even  the  definition  of  many  of  these  terms  are  standardized! 
The  computing  oermunity  is  then  faced  with  the  conflicting  desires  of 
continuing  the  development  of  hardware-software  systems  in  an 
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unfettered  techno  logical  environment,  while  at  the  sane  time,  heirg 
able  to  utilize  each  other's  developments  without  extensive  reprogiam- 
minq  efforts. 

Computer  networks  offer  a  solution  to  this  dilerma  to  the 
extent  that  they  offer  a  viable  method  for  interconnecting  inavpatible 
systems,  and  thereby  allow  resources  to  be  shared  over  a  wider  range  of 
users  without  having  to  standardize  beyond  the  acceptance  of  a  ocmmon 
set  of  interface  conventions  for  hardware,  programs,  and  data.  Such  a 
network  also  allows  the  user  ocrmunity  to  share  other  resources  such  as 
data  bases,  large  computing  power  facilities,  or  specialized  hardware 
systems,  and  in  seme  cases  to  provide  load-sharing  and  hack  up  relia¬ 
bility.  For  noire  research  oriented  facilities,  networks  may  provide 
the  mechanism  for  utilizing  and  building  upon  the  work  of  others,  and 
eventually  to  allow  as  to  correct  the  following  situation  as  quoted 
from  Hamming  (HA69)  in  regard  to  the  lack  of  "science"  in  computer 
science , 

"Indeed,  one  of  my  major  complaints  about  the  oemputer 
fiell  is  that  whereas  Newton  could  say  'If  I  have  seen  a 
little  further  than  others  it  is  because  I  have  stood  on 
the  shoulders  of  giants',  I  am  forced  to  say,  'Today  we 
stand  on  each  other's  feet.'  Perhaps  the  central  problem 
we  face  in  all  of  oemputer  science  is  hew  we  are  to  get 
to  the  situation  where  we  build  on  top  of  the  work  of 
others  rather  than  redoing  so  much  of  it  in  a  trivially 
different  way.  Science  is  supposed  to  be  cumulative,  not 
almost  endless  duplication  of  the  sane  kind  of  things." 

Ihe  ARPA  network  ocrmunity*  has  already  shewn  a  refreshing  willingness 

to  work  together  to  solve  the  problems  of  networking  and  to  work 

The  ARPA  (Advanced  Research  Projects  Agency)  network  ocrmunity  con¬ 
sists  of  about  twenty  different  research  oenters  across  the  country.  A 
description  of  the  facilities  in  the  network  is  given  in  Section  1.3. 
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towards  the  cumulative  development  effort  to  which  Hanning  referred. 
This  cooperative  spirit  and  improved  communications  may  provide  an  on- 
virorment  suitable  to  bring  together  what  Roberts  (RD67)  calls  a 
"critical  mass"  of  interdisciplinary  talents  and  resources  to  solve 
problems  which  heretofore  were  beyond  the  realm  of  solution.  In  addi¬ 
tion,  the  entire  network  is  being  considered  as  an  experiment,  and  is 
being  modeled  and  evaluated  (both  analytically  and  by  direct  measure¬ 
ment)  to  determine  how  well  it  meets  its  design  goals,  and  to  investi¬ 
gate  areas  of  future  improvement. 

This  dissertation  is  primarily  concerned  with  the  measurement 
aspects  of  the  network  evaluation,  and  considers  the  design  of  a  set  of 
measurement  facilities,  the  development  of  data  gathering  and  reduction 
techniques,  and  the  usage  of  these  measurements  to  evaluate  the  network 
performance.  The  latter  will  include  comparisons  of  measured  results 
with  the  predictions  of  analytic  models  which  were  developed  for  this 
study.  However,  before  discussing  the  measurement  considerations,  we 
will  review  some  earlier  efforts  at  network-related  analytic  and  simu¬ 
lation  model  developments. 

1.1  Evolution  of  Computer  Networks 

Computer  networks  are  a  natural  part  of  the  evolution  of 
telecarmunication  oriented  computer  systerus.  These  systems  originated 
with  the  utilization  of  telephone  circuits  for  batch-oriented  data 
transfers ,  and  blossomed  due  to  the  need  for  remote  computer  access  for 
time-sharing  and  remote- job-entry  applications.  These  remote  terminals 
were  often  made  more  sophisticated  by  the  introduction  of 
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pre-processing  haru..are,  terminal  multiplexing  functions,  display 
refresh  requirements,  or  off-line  data  collection  and  minor  processing 
functions,  and  hei-jce  began  to  distribute  the  processing  functions 
across  such  a  network.  However,  it  was  not  until  the  ARPA  network  ex¬ 
periments  that  autonomously  operating  processors  were  interconnected 
for  resource  sharing  functions.  The  first  APPA  experiment,  involved  the 
experimental  interconnection  of  a  small  Control  Data  Corporation  16HA 
computer  at  SRI  (Stanford  Research  Institute)  to  the  large  Q-32  com¬ 
puter  at  SDC  (System  Development  Corporation)  in  1964.  These  tests  led 
to  the  interconnection  of  two  large  processors,  the  0-32  at  SDC  and  the 
TX-2  computer  at  the  Lincoln  Laboratories  in  1966  and  1967.  The  ex¬ 
perimental  "network"  was  described  by  Mari  11  and  Roberts  (MA66) ,  and 
demonstrated  the  utilization  of  two  large  autonomously  operating  proc¬ 
essors  in  a  network  configuration,  and  also  provided  a  test  bed  for 
developing  an  intercommunication  protocol,  for  evaluation  of  telephone 
communication  facilities ,  for  integration  of  networking  functions  into 
the  two  time-sharing  rronitors ,  and  for  demonstrating  the  usage  of  pro¬ 
grams  at  one  facility  as  subroutines  to  the  other  computer.* 

The  Q-32 /TX-2  "network"  utilized  conventional  direct-dial 
telephone  facilities  for  the  inter-processor  communications ,  but  these 
corrrruni cation  facilities  were  found  to  be  unsatisfactory  due  to  the 

*0ne  exarple  usage  of  the  experimental  TX-2  to  Q-32  net  involved  the 
use  of  a  Q-32  LISP  program  at  SDC  to  perform  an  infix- to-prefix  trans¬ 
lation  of  a  source  program  sent  from  the  TX-2  at  Lincoln  Laboratories. 
The  Q-32  would  then  return  the  translated  form  back  to  the  TX-2  where 
the  program  would  be  executed.  In  this  manner,  the  Q-32  program  was 
utilized  as  a  subroutine  to  the  TX-2  although  they  were  different 
machines,  separated  by  several  thousand  miles,  ran  under  different  op¬ 
erating  systems,  and  utilized  different  programming  languages. 
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time  required  to  form  the  connection  at  each  data  transfer.  The  only 
alternative  was  to  maintain  the  connection  for  the  duration  of  the 
experiment,  ard  therefore  to  utilize  +-he  line  very  inefficiently. 

These  problems  lead  to  the  development  of  a  more  oost-effective  store- 
and-forward  oomnuni cation  system  for  the  more  lecent  ARPA  network. 

Such  a  system  is  referred  to  as  a  message  switching  system  as  opposed 
to  the  conventional  line-switching  system  of  the  existing  telephone 
network,  and  required  additional  considerations  of  s tore-and- forward 
delays,  alternate  routings,  and  error  control.  However,  the  message 
switching  system  provides  the  benefit  of  multiplexing  the  usage  of 
higher  bandwidth  lines  so  that  the  bandwidth  can  be  more  efficiently 
utilized,  and  dynamically  selects  the  transmission  paths  based  on  line 
quality  and  congestion.  In  contrast,  line  switching  facilities  dedi¬ 
cate  a  fixed  bandwidth  to  each  user,  and  select  the  transmission  path 
via  an  electronic  switch  mechanism  at  the  time  the  connection  is  first 
made. 

The  choice  between  line  and  message  switching  depends  on  both 
technological  advances  and  tariff  changes,  such  that  the  optimal  selec¬ 
tion  between  the  two  techniques  may  vary  with  time.  The  communication 
networks  being  oroposed  by  MCI  (Micrcwave  Comunications  Incorporated) 
and  DATRAN  may  also  impact  these  considerations  as  may  the  use  of  PCM 
(pulse  code  modulation)  techniques  by  the  existing  cannon  carriers, 
since  the  higher  bandwidth  of  the  PCM  system  oculd  be  available  for 

direct-dial  data  tranarussion.*  Hcwever,  at  the  present  time,  the  use 
I - 

Further  information  on  telecommunications  systems  can  be  found  in 
Martin  (MA69) ,  Hansher  (HA68) ,  and  Hersch  (HE71). 
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of  message  switching  is  very  cost  effective,  with  a  cost  reduction  of 
two  orders  of  magnitude  being  shown  by  Roberts  and  Wessler  (RO70)  for 
the  case  of  sending  a  megabit  of  information  between  two  average  net¬ 
work  sites. 

1.2  Other  Computer  Networks 

Before  proceeding  with  a  discussion  of  other  networking 
attempts,  it  is  appropriate  to  define,  or  at  least  delimit,  our  meaning 
of  "ccnputer  networks".  A  precise  definition  is  of  little  consequence 
since  we  hope  that  many  of  our  measurement  and  modeling  techniques  will 
be  useful  to  a  wide  variety  of  teleocnmanicaticn  and  computer  systems. 
However,  in  the  interest  of  delimiting  the  scope  of  the  study,  we  offer 
the  following  definition  of  a  ccnputer  network.  "A  ccnputer  network  is 
a  set  of  inter  connected  processors  which  can  he  utilized  jointly  in  a 
productive  manner,  but  which  normally  are  controlled  by  separate  oper¬ 
ating  systems,  and  can  perform  in  an  autonomous  manner."  With  this 
description  serving  more  as  a  guideline  than  definition,  we  will  new 
consider  sane  other  network  systems  and  experiments . 

One  of  the  earliest  attempts  at  integrating  a  network  of 
oonputers  was  the  SAGE  (Semi -Automatic  Ground  Environment)  system  which 
was  developed  to  collect,  analyze,  and  display  radar  data  trcm  sensors 
scattered  over  the  continent  (MA69) .  The  systen  became  operational  in 
1958,  and  lias  subsequently  been  updated  and  improved.  About  that  same 
time,  the  American  Airlines  SABFE  reservation  system  was  being  devel¬ 
oped  on  a  carmercial  basis  (EV67) ,  and  its  success  has  lead  to  the  use 
of  similar  systems  by  other  airlines,  car  rental  agencies,  hotels,  etc. 
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The  need  for  improved  data  transmission  facilities  led  the 
military  to  development  of  the  AUTODIN  (Automatic  Digital  Network)  data 
oormunications  system  in  1963  (UA6R) .  The  system  includes  both  message 
and  line  switching  facilities,  and  was  designed  with  considerable 
thought  being  given  to  the  network  survivability  and  vulnerability.  In 
contract  to  this  military  need  for  ultra-reliability,  cemnercial  and 

r  * 

experimental  networks  have  utilized  relatively  simple  interconnection 
patterns  or  have  relied  on  the  direct-dial  telephone  network  for  com¬ 
munications .  Examples  of  such  systems  include  the  Control  Data  Cor¬ 
poration  Cybernet  system  (BE69)  and  the  experimental  CMJ-Princeton-IEM 
network  (RU69B)  respectively.  The  latter  net  consisted  of  three 
IBM  360/67  computers,  each  operating  \inder  the  TSS/67  time  sharing 
monitor.  Both  networks  are  typical  of  most  earlier  networking  attempts 
in  the  similarity  of  machine  types  and  operating  systems  within  any 
given  net. 

Several  networks  have  been  designed  using  a  central  store- 
and- forward  message  switch,  which  can  reduce  the  network  cost,  but  of 
course  increases  the  vulnerability  of  the  network  to  the  loss  of  the 
central  node.  Such  a  topology  has  been  utilized  for  the  COINS  (Can- 
munity  On-Line  Intelligence  Network  System),  the  Lawrence  Radiation 
Laboratory  OCTOPUS  system,  and  a  proposed  IBM  network.  The  COINS  sys¬ 
tem  was  described  at  the  1968  Computer  Network  Workshop  (GE69) ,  al¬ 
though  little  detail  was  given  due  to  the  military  intelligence  nature 
of  the  system.  It  appeared  to  be  intended  as  a  shared  data  base  system, 
and  utilized  a  standard  i.zed  ASCII  character  code  and  a  MIL-STO-188B 
hardware  interface  to  simplify  the  interconnections. 
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The  Laurence  Radiation  Laboratory  network  was  called  OCTOPUS 
due  to  its  star-like  topology,  and  was  described  by  Fletcher  at  this 
same  workshop.  The  central  computer  is  a  PDP-6  which  serves  as  a 
store- and- forward  switch  between  the  large  processors  such  as  CDC  6600, 
7600,  and  STAR,  as  well  as  the  IBM  Stretch  and  360/91  canputers.  The 
central  switch  also  provides  access  to  the  huge  photo-store  mass  memory 
by  any  of  the  other  machines,  and  allows  an  evolutionary  growth  of  the 
multi-conputer  complex  since  new  computers  can  be  connected  to  the  sys¬ 
tem  resources  and  can  gradually  be  brought  up  to  operational  status. 

The  third  star  network  is  the  IBM  computer  network*  which  has 
several  unusual  features .  The  central  node  was  initially  to  be  a  radi¬ 
um  size  360/50  computer,  but  was  later  changed  to  be  a  partition  in  the 
large  360/91,  which  serves  not  only  as  a  store-and-forward  switch,  but 
also  as  a  master  operating  system.  The  network  will  consist  of  several 
IBM  360  ccrputers  and  a  Control  Data  6600  computer,  the  latter  being 
connected  via  a  small  Honeywell  DDP-516  preprocessor.  The  non-IIM 
machine  will  introduce  a  degree  of  generality  into  the  network  due  to 
the  considerable  difference  in  the  CDC  and  IEM  architecture  and  data 
structures. 

Several  other  networks  or  proposed  networks  utilize  multiply 
interconnected  topologies,  much  like  that  utilized  for  the  ARPA  network. 
The  most  similar  was  a  network  proposed  by  the  National  Plyuical  Lab¬ 
oratory  in  England  and  described  by  Davies,  et  al.  (DA67  &  68), 

*7he  IEM  network  will  be  described  in  a  future  ACM  aonference  paper  by 
D.  McKay  and  D.  Karp,  entitled,  "Network/440  -  IEM  Research  Computer 
Sciences  Department  Carputer  Network". 
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although  it  has  apparently  not  developed  beyond  the  proposal  stage.  It 
was  first  described  in  this  country  at  the  Oat  1  inburg  conference  on 
operating  systems,  concurrent  with  Roberts’  description  of  the  preposed 
ARPA  network  (R067) ,  which  was  an  extension  of  his  previously  described 
pioneering  work.  The  NPL  network  was  to  be  a  store-and- forward  network 
using  interface  computers  and  1.5  Mb/sec.  transmission  lines  for  the 
message  switching  net,  with  an  expected  network  response  time  (the  time 
from  the  receipt  of  a  padeet  to  the  beginning  of  the  output  at  the  des¬ 
tination)  of  less  than  100  msec.  Packets  were  defined  as  any  multiple 
of  128  bit  segments  up  to  a  maximum  of  1024  bits.  The  NPL  network 
design  differed  from  that  of  the  ARPA  net  in  the  choice  of  acknowledge¬ 
ments  (negative  acknowledgements  arid  "stop  sending"  messages  in  the  I  PL 
design  ccqpared  to  nositive  acknowledgements  in  the  ARPA  net) ,  in  the 
use  of  a  handover  nuttier  in  the  NPL  design  to  discard  messages  in  the 
case  of  excessive  loopim,  and  in  the  use  of  a  check  sum  with  each  128 
bit  segment  while  the  ARPA  net  utilizes  a  24  bit  cyclic  check  code  with 
each  packet.  Other  features  of  the  ARPA  net  will  be  described  in 
Section  1.3. 

A  small  experimental  computer  network  is  being  developed  at 
Carnegie  Mellon  University  (BE70A) ,  consisting  of  two  DEC  PDP-10  com¬ 
puters,  a  pair  of  PDP-8  miniccrrputers ,  and  a  hybrid  computer.  All  five 
ermputers  will  be  located  together,  and  since  the  ccrtmini cations  costs 
will  be  insignificant,  they  plan  to  experiment  with  completely  con¬ 
nected  nets  as  well  as  with  more  typical  network  interconnection  topol¬ 
ogies.  They  also  plan  to  investigate  various  aspects  of  operating 
systan  control.  Since  CMU  will  also  be  a  node  in  the  ARPA  network, 
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tliis  may  be  one  of  several  local  sub  nets  within  the  overall  network. 

A  network  whidi  is  considerably  different  from  those  dis¬ 
cussed  above  is  the  National  Crime  Information  Center  (NCIC)  network 
which  connects  law  enforcement  data  retrieval  systems  across  the  nation 
into  the  Federal  Bureau  of  Investigation  data  bank  (GE69).  Many  states 
have  expanded  these  facilities  to  include  other  data  bases  such  as  de¬ 
partment  of  motor  vehicles  records  and  local  police  file  systems.  The 
resulting  network  is  quite  complex,  but  has  been  reasonably  successful 
due  to  the  ability  of  eadi  systan  to  simulate  the  terminal  character¬ 
istics  and  inputs  of  any  other  system  being  accessed.  This  was  pos¬ 
sible  due  to  rigid  format  definitions  and  a  reasonably  small  number  of 
differing  terminal  types. 

Numerous  other  networks  are  presently  in  the  design  stage, 
and  when  implemented,  will  include  several  additional  large  univer¬ 
sities.  New  network  structures  are  also  being  investigated  with  the 
goal  of  providing  more  cost-effective  data  transmission  facilities;  one 
such  exarple  being  the  ring  structure  proposed  by  Pierce  (PI71) .  The 
virtual  explosion  of  interest  in  networking  should  produce  significant 
changes  in  the  way  in  which  computers  are  utilized,  and  will  add  a 
valuable  new  set  of  resources  to  each  computer  center  involved  in  such 
a  network  environment. 

1.3  The  ARPA  Computer  Network 

The  ARPA  network  of  cxmputers  is  by  far  the  most  ambitious 
network  effort  ever  undertaken,  both  in  terms  of  the  variety  of  facili¬ 
ties  and  in  the  degree  of  sophistication  of  the  resources  being 
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integrated.  The  basic  ne^ork  ocrmunity  consists  of  about  twenty  ARPA 
sponsored  research  sites,  many  of  which  have  areas  of  specialization 
such  as  the  graphics  work  at  the  University  of  Utah,  the  man-machine 
interaction  work  at  SDC,  the  text  editing  and  information  retrieval 
work  at  SRI  ani  the  network  measurement  and  modeling  work  at  UCIA. 

Some  other  nodes  have  special  hardware  including  the  ILLIAC  IV  com¬ 
puter,  the  trillion  bit  laser  memory,  and  perhaps  later,  special  micro- 
programing  and  list  processing  equipment.  The  combination  of  these 
hardware  and  software  resources  and  the  cooperating  interdisciplinary 
human  resources  at  the  various  sites,  creates  a  viable  environment  for 
new  research  in  computer  related  fields. 

The  various  sites  are  interconnected  via  a  distributed  store- 
and- forward  cormini cation  net  consisting  of  IMP's  (Interface  Message 
Processors)  and  50  Kbit/sec.  full  duplex  ccmnuni cation  lines.  Each  site 
typically  consists  of  one  or  more  HOST  computers  operating  in  a  time¬ 
sharing  environment,  but  range  in  size  frcm  a  terminal  PIP  (a  partition 
within  the  IMP  itself)  to  the  very  large  ILLIAC  IV.  More  typical  com¬ 
puter  facilities  are  in  the  Digital  Equipment  Corporation  PDP-10  or  the 
IBM  360/67  class,  but  often  include  special  terminal  or  graphics  equip¬ 
ment  as  well. 

Each  IMP  connects  to  its  local  HDST(s)  and  typically  to  three 
or  four  ccmnuni  cation  lines  as  shown  in  Figure  1.3.1.  The  IMP  is  it¬ 
self  a  small  computer,  a  Honeywell  DDP-516,  with  12  K  16-bit  words  of 
memory,  special  ocrminication  interface  circuits,  and  other  features 
as  needed  to  nerform  the  message  switching  task.  However,  it  does  not 
include  any  disk  or  tape  storage  facilities  nor  any  redundant 
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Figure  1.3.1.  The  ARPA  Network  Configuration  at  of  Mid-1971 


capability  for  back-up  operation.  The  software  for  the  IMP  functions 
consists  of  a  set  of  interrupt  driven  routires  which  handle  the  real 
time  I/O  functions  and  create  tasks  which  are  then  queued  and  scheduled 
according  to  their  relative  priorities  and  need  for  output  channels  or 
other  resources.  A  nunber  of  lew  priority  background  programs  are  also 
included  in  the  IMP  functions,  and  are  considered  to  be  special  "Fake 
HOST'S"  since  they  can  be  the  source  aid/or  destination  of  messages. 
These  functions  include  the  IMP  console  Teletype,  the  debug  package  and 
the  measurement  facilities.* 

HOST-IMP  transmission  are  in  terms  of  physical  records  called 
"messages",  which  consist  of  a  32-bit  leader  followed  by  up  to  8064 
bits  of  data.  These  messages  are  segmented  into  smaller  "oackets"  for 
the  store-and-forward  transmission  through  the  net,  with  each  packet 
consisting  of  a  64-bit  header  and  up  to  1008  bits  of  data.  The  packets 
are  reassembled  at  the  destination  IMP  with  the  IMP-to-HOST  transmis¬ 
sion  being  in  terms  of  a  message  length  record.  However,  within  the 
network,  the  flow  is  in  terms  of  packets,  with  each  packet  being  saved 
until  a  successful  acknowledgement  has  been  received  from  the  next  IMP 
along  the  store-and-forward  path.  The  next  step  in  the  path  is  se¬ 
lected  at  each  IMP  from  its  own  routing  table  which  is  updated  about 
every  ha  If -second  based  on  nearest-neighbor  routing  information  as 
described  by  Fultz  and  Kleinrock  (FU71) .  The  successful  receipt  of  a 
packet  is  based  on  the  agreement  of  a  24-bit  cyclic  check  code  and  the 

availability  of  storage  space,  and  upon  acceptance,  it  beccmes  the 
_ 

More  detailed  information  on  the  Interface  'lessage  Processors  can  be 

found  in  the  paper  by  Heart,  et  al.  (HE70) . 
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responsibility  of  that  IMP  to  ensure  the  successful  store-and-forward 
transmission  along  the  next  step  of  the  path.  The  combination  of  error 
control  and  alternate  routing  makes  the  network  a  very  reliable  trans¬ 
mission  media. 

Congestion  is  a  potential  problem  in  any  store-and-forward 
net  because  the  peak  data  rates  can  easily  exceed  the  instantaneous 
channel  capacity.  However,  severed  congestion  control  features  were 
built  into  the  ARPA  network  to  minimize  the  consequences  of  overload 
conditions  including;  (1)  the  use  of  logical  links  for  cormtuni  cation 
between  programs  at  separate  HOOT's,  with  the  links  becoming  blocked 
until  a  RFNM  (Request  For  Next  Message)  has  been  received,  (2)  the  use 
of  adaptive  alternate  routing,  (3)  present  limits  an  IMP  buffer  alloca¬ 
tion  maxima  for  reassembly  and  store-and-forward  packets,  and  (4)  the 
use  of  time-outs  to  discard  undeliverable  messages  or  message  fragments . 

The  problems  of  congestion  and  message  delays  are  seme  of  the 
more  interesting  aspects  of  such  a  network  frem  a  measurement  and  mod¬ 
eling  viewpoint,  and  will  be  pursued  at  length  in  subsequent  portions 
of  this  study.  Such  models  typically  involve  queues  in  which  messages 
or  packets  are  delayed,  and  in  the  more  general  case  involve  networks 
of  queues,  both  within  the  store-and-forward  switches  and  across  the 
global  aspect  of  the  caimuni cation  net.  Since  much  of  the  measurement 
and  node  ling  work  involves  queueing  theory  and  queueing  related  vari¬ 
ables  such  as  the  arrival  times,  service  times,  waiting  time,  mmber  in 
the  system,  etc.,  it  is  appropriate  to  review  the  state  of  development 
of  this  portion  of  queueing  theory. 
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1.4  Analysis  and  Modeling  of  Networks* 

The  analysis  of  queueing  systems  has  reached  a  high  degree  of 
applicability  ard  mathematical  sophistication  in  recent  years.  However, 
the  problems  of  tandem  queues  or  networks  of  queues  have  been  found  to 
be  un tractable  except  for  certain  special  cases,  due  to  the  inter¬ 
actions  between  the  various  queues,  and  the  resulting  dependencies 
which  ccmplicate  the  analysis. 

Burke  (BU56)  made  an  initial  contribution  to  the  analysis  of 
networks  of  queues  by  proving  that  for  a  simple  Markovian  queueing  sys¬ 
tem,  (i.e. ,  a  system  with  infinite  queueing  space,  exponentially  dis¬ 
tributed  service  times,  and  Poisson  arrivals  with  an  arrival  rate  of 
X),  the  output  process  is  also  Poisson  with  parameter  X.  Thus,  for 
such  queueing  subsystems,  there  are  no  dependencies  and  the  analysis 
can  be  handled  in  a  reasonably  straightforward  manner.  J.  R.  Jackson 
(JA57)  utilized  this  approach  for  the  analysis  of  a  network  of  queues, 
in  which  each  node  is  an  independent  multiserver  subsystem  with  ex¬ 
ponential  service,  and  with  arrivals  from  both  within  the  net  and  frcm 
the  outside,  (but  with  all  inputs  forming  a  composite  Poisson  stream). 

This  seminary  of  developments  in  the  analysis  of  networks  of  queues  is 
rather  cursory,  but  is  intended  to  give  an  appreciation  for  the  prob¬ 
lems  involved  in  the  analysis  of  interconnected  queueing  systems.  For 
more  information  on  these  topics,  the  reader  is  referred  to  Reich 
(RE57)  and  Finch  (FI 59)  which  define  necessary  and  sufficient  condi¬ 
tions  for  Burke's  result,  and  for  the  analysis  of  sequential  arrays  of 
queues  with  finite  storage,  to  Hunt  (HU56) ,  K)einrock  (KL65)  and  Cox 
and  Smith  (0061) .  Simulation  results  have  been  described  by  Kleinrock 
(KL64  and  KL70)  and  by  Wallace  (WA66) ,  and  have  included  performance 
evaluation  as  well  as  topological,  routing,  and  channel  capacity  as¬ 
signment  considerations.  Recent  work  in  the  analysis  of  closed  queue¬ 
ing  systems  using  the  results  of  Gordon  and  Newell  (G067) ,  is  also 
relevant  to  network  analysis. 
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However,  this  analysis  did  not  consider  che  case  of  message  transmis¬ 
sion  through  a  network  which  requires  that  tlv^  service,  i.e.,  serial 
transmission,  be  a  fixed  value  at  each  subsequent  serve:.  Kleinrock 
(KL64)  recognized  this  problem  and  concluded  that  the  only  tractable 
analytic  approach  was  to  assume  that  the  message  length  was  an  exponen¬ 
tial  randan  variable,  to  be  selected  independently  at  each  server, 
i.e.,  the  "independence  assumption".  His  original  model  also  assumed 
noise  free  channels,  reliable  nodes,  single  destination  messages,  no 
defections,  infinite  buffering,  stationarity  of  the  stochastic  proc¬ 
esses,  fixed  routing,  negligible  nodal  delays,  and  the  lack  of  induced 
traffic  such  as  response  messages,  but  his  more  recent  results  (KL69B) 
have  included  the  latter  two  effects  and  have  approximated  the  effect 
of  non-exponential  service.  Other  work  at  UCIA  and  other  research 
sites  have  utilized  simulation  methods  to  gain  insights  into  the  queue¬ 
ing  behavior  of  such  complicated  networks,  with  the  combination  of 
analytic  and  simulation  techniques  being  complementary  in  many  cases. 

There  have  also  been  several  recent  developments  in  the  de¬ 
sign  of  optimal  (or  "sub-optimal")  network  topologies  and  channel  capa¬ 
city  assignments .  These  problems  typically  defy  a  precise  optimal 
solution  but  good  selection  techniques  have  been  described  by  Frank, 
et  al.  (FR70)  and  Kleinrock  (KL70)  respectively. 

Implicit  in  any  optimal  network  design  are  one  or  more  per¬ 
formance  rreasures  which  have  been  minimized  (or  maximized) ,  and  the 
careful  selection  of  these  performance  criteria  is  quite  important 
since  any  subsequent  optimization  is  based  on  satisfying  these  primary 
goals.  Kleinrock  (KL64)  chose  the  average  message  delay  as  his 
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performance  measure  for  a  store-and- forward  net,  and  analogous  response 
time  performance  measures  have  been  utilized  in  time-sharing  analysis 
and  other  job  turn-around  time  studies.  These  measures  meet  the  gen¬ 
eral  requirements  of  a  performance  metric,  nanely  they  have  a  physical 
meaning  or  interpretation,  are  quantitative,  and  are  an  indicator  of 
the  system  effectiveness.* 

The  close  tie  between  (1)  performance  measures,  (2)  analytic 
and  simulation  models,  and  (3)  measurenents  was  brought  out  by  Estrin 
and  Kleinrock  (ES67B)  in  a  paper  titled  "Measures,  Models  and  Measure- 
rents  for  Time-Shared  Computer  Utilities".  The  combination  of  these 
factors  is  felt  to  be  basic  to  the  scientific  method  of  investigation, 
and  will  be  stressed  throughout  this  study,  although  our  main  focus 
will  be  on  the  measurement  aspects  of  the  network  evaluation. 

1.5  The  Measurement  Program  Associated  With  the  ARPA  Network 

Performance  reasuranent  of  the  ARPA  network  has  been 
considered  to  be  an  integral  part  of  the  design  requirements  from  the 
early  stages  of  the  network  development.  This  concern  for  measurements 
is  due  to  several  factors  including  the  desire  to  gain  insights  into 
network  behavior,  to  aid  in  modeling  ard  analysis,  to  evaluate  the  net¬ 
work  design,  ard  to  indicate  areas  where  redesign  may  be  needed.  In 
sane  cases,  the  measurerents  may  even  indicate  design  parameters  which 
can  not  be  adequately  selected  as  a  constant  value,  but  rather  should 
dynamically  vary  depending  on  the  network  state  or  traffic  load.  Such 
par  are  ter  s  would  then  beccne  functions  of  the  measured  values ,  similar 

Adapted  from  Goode,  H.  and  R.  Machol,  System  Engineering,  McGraw-Hill, 

1957,  pp.  125-126. 
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to  the  present  routing  table  dynamic  updating.  However,  our  initial 
concerns  are  primarily  those  of  performance  evaluation  and  gaining  in¬ 
sight  into  critical  performance  par  are  ter  s. 

1.6  A  Statement  of  the  Problem  Being  Considered 

UCIA  is  actively  involved  in  the  measurement  and  analysis  of 
the  ARPA  network  behavior,  and  as  the  Network  Mea-urement  Center  (NMC) , 
has  the  responsibility  of  defining  the  measurement  techniques,  and 
performing  appropriate  network  measurements  and  experiments.  These 
efforts  closely  follow  Hamming's  philosophy*  that  "the  purpose  of 
measurements  is  insight,  not  numbers  I " ,  with  the  measurement  results 
being  utilised  in  support  of  the  analytic  and  simulation  modeling  ef¬ 
forts,  the  determination  of  network  performance,  and  to  determine  any 
areas  in  which  network  improvements  need  to  be  made. 

The  overall  measurement  problem  can  be  described  in  several 
somewhat  overlapping  tasks,  including: 

1.  Identification  of  the  measurement  needs  for  modeling, 
simulation,  performance  evaluation,  user  behavior  meas¬ 
urement,  and  possible  areas  of  dynamic  control. 

2.  Development  of  new  data  gathering  techniques  as  required 
by  the  network  environment  and  constraints. 

3.  Design  of  a  set  of  measurement  routines  of  sufficient 
power  and  generality  to  be  useful  for  an  open-ended  set 
of  experiments. 

An  often  paraphrased  quote  attributed  to  R.  Harmung. 
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4.  Provide  test  cases  by  developing  several  analytical 
models  for  experimental  evaluation. 

5.  Empirical  investigation  of  the  usefulness  of  the  measure¬ 
ment  facilities  in  both  model  verification  and  in  explor¬ 
atory  data  gathering  experiments,  which  might  then  pro¬ 
duce  new  insights  and  results  in  an  iterative  sequence 
of  experiments. 

These  tasks  are  felt  to  be  necessary  to  provide  an  extensive' 
measurement  capability  of  the  ARPA  network  performance,  and  to  overcome 
the  limitations  observed  by  Estrin  and  Kleinrock  (ES67B) ,  that 
"...present  methods  of  measurement  do  not  allcw  sufficient  freedom  in 
design  of  experiments  on  complex  systems".  And  "Most  of  the  weaknesses 
in  the  relevancy  of  measurements  to  models  arises,  at  present,  from  the 
unavai lability  of  bools  for  making  desired  observations  of  dynamic 
systons".  Hopefully,  the  network  measurement  techniques  have  avoided 
these  past  limitations,  and  will  be  sufficient  to  meet  the  needs  of 
future  experiments,  arci  will  provide  the  necessary  feedback  to  evaluate 
the  continuing  usage  of  the  ARPA  network. 
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CHAPTER  2 


DEVELOPMENT  OF  A  SET  OF  NETWORK  MEASUREMENTS 

Tiie  desire  to  measure  and  evaluate  the  network  performance 
has  been  one  of  the  goals  of  the  ARPA  network  since  early  in  its  con¬ 
ception,  but  because  there  has  been  little  precedence  in  this  area  of 
rreasurement,  we  found  it  necessary  to  consider  a  ranter  of  .aspects  of 
the  neasurement  needs  and  related  costs  in  the  network  environment. 

This  investigation  included  a  survey  of  measurement  techniques  in  re¬ 
lated  oonputer  applications,  considered  sane  of  the  more  fundamental 
measurement  concepts,  attainted  to  identify  the  areas  in  which  measure¬ 
ment  data  were  needed,  and  ended  in  the  design  of  a  set  of  measurement 
tools  for  the  network  study. 

2.1  A  Survey  of  Related  Measurement  Efforts 

Although  few  previous  network  measurements  have  been  made, 
there  were  seme  data  gathering  facilities  included  in  the  earlier  ex¬ 
perimental  links  between  the  Q-32  ocmputer  at  SDC,  the  TX-2  oonputer  at 
Lincoln  Laboratories ,  and  the  DEE  338  display  computer  at  ARPA.*  These 
results  were  quite  dependent  upon  the  particular  programs  being  utilized 
in  the  experimental  net,  and  were  limited  to  measurements  of  the  number 
of  calls,  the  call  durations,  the  number  of  messages  sent,  and  the  num¬ 
ber  of  characters  sent.  Seme  data  was  also  taken  on  the  COINS  network 
(GE69)  by  use  of  a  program  which  periodically  started  a  pseudo-user  job 

As  reported  in  "An  Experimental  Conputer  Network",  Computer  Corpora¬ 
tion  of  America  Report  NO.  ESD-TR69-74  (AD694-055),  March  1969. 

Preceding  page  blank 
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and  measured  the  response  time.  Earlier  work  in  carputer  system 
measurements  has  developed  a  variety  of  hardware  and  software  rc<.ording 
techniques.  These  efforts  can  be  categorized  .into  three  broad  classes; 
(1)  measurements  of  time-sharing  system  behavior,  (2)  measurements  of 
user-oorputer  interactions,  and  (3)  measurements  of  program  behavior. 

We  shall  sunmarize  seme  of  these  results  which  relate  to  the  network 
measurements  in  the  following  paragraphs. 

Measurements  were  taken  on  the  original  time-sharing  systems 
and  were  reported  by  Scherr  (9Ctf7A)  and  Totschek  (T065)  on  the  MIT 
Project  MAC  Compatible  Time- Shared  System  (CTSS)  and  on  the  System 
Development  Corporation  0-32  system  respectively.  Scherr 's  measure¬ 
ments  were  related  to  analytic  models  which  he  developed  and  included 
the  effects  of  differing  time  quanta  and  rnrnber  of  users,  as  well  as 
Pleasuring  distributions  of  user  think  time,  program  sizes,  and  proces¬ 
sing  requirements.  Tots  check  measured  the  distributions  of  service  and 
interarrival  times,  the  effect  of  the  nurber  of  simultaneous  users,  and 
the  system  overhead,  and  the  utilization  of  the  CPU,  dram  and  core 
memory.  Many  of  the  empirical  distributions  were  found  to  have  similar 
characterise  re  including  a  range  that  covered  several  orders  of  magni¬ 
tude,  with  density  functions  that  displayed  long  slcwly  decreasinc 
tails,  and  resulted  in  a  standard  deviation  exceeding  the  mean  value. 

He  also  observed  that  other  data  appeared  to  be  from  a  non-hemogeneous 
population,  and  assured  that  the  underlying  population  was  actually  a 
set  of  different  sub-groups. 

In  both  of  these  measurement  efforts,  the  data  were  obtained 
as  a  result  of  nodifications  to  the  operating  system  programs  to  record 
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appropriate  times  and  events  for  later  reduction,  although  Scherr 
discussed  the  feasibility  of  statistical  reduction  of  the  data  as  it  was 
gathered.  In  aontrast  to  these  software  techniques  of  measurement,  one 
can  utilize  a  hardware  data  gathering  approach  as  described  by  Schulman 
(9C67B).  This  device,  called  a  Time-Sharing  System  Performance  Activ¬ 
ity  Recorder  (TS/SPAR) ,  was  able  to  record  up  to  25f  internal  signals 
of  the  IBM  360/67  CPU,  registers,  and  charnels  which  were  being  physi¬ 
cally  monitored  by  electrical  probes.  The  data  was  then  written  on  a 
separate  magnetic  tape  which  was  part  of  the  hardware  monitor.  Similar 
monitors  have  beucme  commercially  available  in  recent  years ,  as  have 
special  software  monitoring  packages .  Both  have  strengths  and  limita¬ 
tions,  and  as  usual,  a  suitable  mixture  of  such  hardware  and  software 
techniques  is  often  found  to  be  efficient.  A  proposed  experiment  which 
utilized  such  a  combination  of  techniques  was  the  SNUPER  computer 
(ES67A)  wnich  was  designed  to  monitor  the  operation  of  a  larger  proces¬ 
sor  by  the  use  of  hardware  n robes,  but  with  data  selection  instants 
being  determined  by  special  calls  that  had  been  imbedded  in  the  program 
being  executed.  However,  a  second  mode  of  operation  was  included  in 
the  design  to  eliminate  the  need  to  such  program  modifications  by  having 
the  monitor  computer  sample  the  state  of  the  larger  machine  at  aopropri- 
ate  times.  This  latter  mode  of  operation  is  complicated  by  difficulty 
in  recognizing  the  desired  states  in  the  object  machine. 

The  location  of  the  measurement  "probes"  and  the  sample 
timing  is  a  non- trivial  task  in  both  the  hardware  and  software  ap¬ 
proaches,  but  can  be  simplified  in  both  cases  if  performance  monitoring 
is  considered  in  the  early  system  design  phases.  For  example,  the 
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software  measurements  on  the  American  Airlines  SABRE  reservation  system 
(EV67)  were  reported  to  have  been  ccrplicated  by  the  need  to  modify 
many  of  the  programs  to  obtain  suitable  measurement  points.  The  author 
strongly  reocmended  that  monitoring  be  included  in  the  design  of  any 
subsequent  system  programs. 

Several  measurements  of  the  user-conputer  interface  have  been 
made  to  determine  the  data  flow,  response  time,  and  ccrruni cation  needs 
for  interactive  systtm  users,  and  include  the  measurements  of  the  JOSS 
system  at  RAND  (BR67) ,  a  comparison  of  three  systems  by  Fuchs,  Jackson, 
and  Stubbs  (FU70  and  JA69) ,  as  well  as  the  previously  described  papers 
by  Scherr  and  Ttotschek.  The  techniques  used  to  measure  and  record  the 
interactions  included  both  hardware  and  software  methods. 

Computer  program  measurements  have  been  made  by  a  nimber  of 
investigators,  and  have  primarily  utilized  software  monitoring  tech¬ 
niques.  The  degradation  of  the  program  running  time  is  not  nearly  as 
important  in  many  ('r  these  cases  since  no  real-time  or  interactive  ap¬ 
plications  are  involved,  and  therefore  the  degree  of  artifact  ranges 
fran  a  few  percent  to  a r.  order  of  magnitude  reduction  in  operating 
speed.  The  latter  cases  are  those  in  which  a  more  detailed  examination 
of  each  program  step  is  required,  and  are  fairly  infrequent  due  to  the 
large  overhead  cost  involved  ir.  obtaining  such  data. 

A  sampling  of  measurements  of  this  category  includes  the 
program  segment  size  study  of  Batson,  et  al.  (EA70) ,  the  measurements 
on  programs  under  the  GECOS  II  operating  system  by  Cantrell  and  Ellison 
(CA68A) ,  which  were  extended  by  Campbell  and  Heffner  (CA68B) ,  the  meas¬ 
urements  on  FORTRAN  programs  by  Russell  and  Estrin  (RLtff'A) ,  the  design 
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of  translator  programs  which  automatically  introduce  suitable  measure¬ 
ment  points  into  object  code  as  suggested  by  Presser  and  Melkanoff 
(PR69) ,  and  the  survey  of  measurements  described  by  Uzgalis  (UZ70) . 

Severed  observations  cam  be  made  about  general  similarities 
in  the  measurement  efforts  described  above.  First,  in  almost  all  cases, 
significant  system  performance  degradations  were  discovered  and  cor¬ 
rected.  A  second  cannon  feature  was  the  frequent  occurrence  of  dis¬ 
tributions  with  exponential-like  shapes,  but  often  with  a  higher  co¬ 
efficient  of  variation  than  the  exponential  density,  and  thirdly,  the 
general  agreement  that  measurement  facilities  should  be  built  into  the 
original  system  design  rather  than  being  an  add-on  or  later  system 
modification. 

2.2  Some  Fundamental  Concepts  of  Measurement 

A  given  digital  system  can  be  defined  by  describing  either 
its  detailed  structure,  i.e. ,  the  hardware  and  software  component  parts 
and  their  interconnection,  or  by  describing  the  system  activity,  i.e., 
the  state  vector  transitions  for  all  possible  system  states  and  input 
stimuli.  Neither  of  these  can  be  aenpietely  measured  or  defined  for 
systems  of  any  significant  complexity,  so  that  measurements  typically 
record  some  subset  of  the  system  behavior,  and  the  experimenter  must 
then  draw  conclusions  or  inferences  from  this  limited  set  of  observa¬ 
tions.  The  art  of  measurement  is  then  to  obtain  the  necessary  informa¬ 
tion  about  the  system  behavior  from  a  relatively  small  set  of  data 
points. 

The  type  of  measurement  information  which  is  desired  will 
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depend  on  the  intended  purpose  of  the  measurements,  and  may  vary 
considerably  depending  on  whether  we  wish  to  determine  if  the  system 
performs  as  specified,  or  to  determine  the  limiting  factors  in  the  sys¬ 
tem  performance.  In  either  case,  the  observed  system  behavior  will  be 
only  a  subset  of  the  state  vector  information,  which  will  be  sampled 
at  seme  subset  of  the  state  transitions.  This  notion  can  be  seen  in 
Figure  2.2.1  which  shows  a  hypothetical  system  activity  diagram  in 
terms  of  the  state  vector  at  sequential  setps  in  time.*  If  we  include 
the  input  stimuli  in  the  state  vector,  then  such  an  activity  diagram 
would  uniquely  define  the  system  behavior  if  the  observations  were  made 
over  a  sufficiently  long  time  interval. 

Measurements  can  be  conveniently  related  to  such  an  activity 
diagram,  with  the  "complete  set"  of  measurement  data  being  the  entire 
activity  record.  More  typically,  one  sanples  the  activity  data  in 
"space"  and  time,  (where  "space"  refers  to  the  state  vector  space) . 

For  example,  the  snap-shot  measurements  which  we  shall  consider  in 
Section  2.5.2  are  a  set  of  selected  state  vector  components  which  are 
recorded  at  800  msec,  time  intervals?  and  therefore  establishes  a  given 
space-time  resolution  level  for  the  resulting  sampled  data.  Some  of 
the  other  measurements  can  be  considered  to  generate  additional  state 
vector  components,  such  as  the  accumulated  counts  of  messages  handled 
or  histograms  of  message  size.  These  new  state  vector  components  would 
be  periodically  read-out  and  reset  to  zero,  and  might  include  counts, 
sums,  max  or  min  values,  most  recent  value,  or  inter-event  times.  In 

This  approach  is  based  on  the  system  activity  theory  of  Klir  (KL67) . 
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Figure  2.2.1.  A  Hypothetical  System  Activity  Diagram  Showing  the 
System  State  Vector  and  the  Measurement  Extension 


seme  other  instances,  the  data  would  be  read-out  based  on  the  occurrence 
of  sane  particular  system  state  (or  event) ,  with  the  two  situations 
being  generalized  if  clock  values  arc  considered  to  bo  part  of  the  state 
vector.  In  fact,  this  assumption  that  the  clock  value  is  part  of  the 
state  vector  allows  us  to  generalize  all  of  our  measurement  techniques 
into  one  carman  activity  representation,  namely  that  seme  subset  of  the 
state  vector  is  to  be  recorded  at  each  occurrence  of  sane  other  state 
vector  component.  Since  we  will  only  be  recording  conponents  of  the 
state  vector  at  the  time  of  the  sample,  any  previous  state  information 
must  be  carried  along  as  an  extension  to  the  basic  state  vector.  For 
example,  if  we  desire  a  count  of  the  number  of  message  inputs,  we  would 
include  an  N-bit  component  in  the  state  vector ,  and  would  increment 
this  field  at  each  message  input.  Other  records  of  past  history  such 
as  histograms,  or  running  sums  would  be  handled  similarly. 

A  more  subtle  measurement  concept  is  that  of  observability  of 
the  desired  state  components.  Since  the  data  is  to  be  taken  upon  the 
occurrence  of  a  predefined  state  vector  component,  we  may  or  may  not  be 
able  to  find  meaningful  data  in  seme  other  ccrponent  of  interest  de¬ 
pending  on  possible  dependencies  of  the  two  components.  For  example, 
if  corponent  A  can  occur  only  when  ccrponent  B  is  equal  to  zero,  and  if 
the  measurement  is  triggered  on  the  occurrence  of  component  A,  then  the 
measured  value  of  B  would  always  be  zero.  Such  a  situation  actually 
occurs  in  the  network  measurements  due  to  the  relative  priorities  of 

the  task  queue  and  the  background  routines. 

Other  aspects  in  the  selection  of  a  suitable  set  of  state 
vector  ccrponent  rreasurements  are  those  of  necessary  sets  of 


28 


measurements  and  sufficient  sets  of  measurements .  For  exanple,  if  we 
are  interested  in  the  behavior  of  three  randan  variables,  x,  y  and  z 
where  we  knew  that  z  =  x  +  y,  it  is  sufficient  to  measure  the  rrean 
value  of  ary  two  to  compute  the  mean  of  the  third,  but  we  need  statis¬ 
tics  on  all  three  (or  exact  dependence  information)  to  obtain  variance 
data  on  the  three  variables.  An  increasing  amount  of  information  is 
therefore  necessary  if  we  have  dependent  randan  variables.  The  ideal 
set  of  measurements  is  then  the  minimal  set  of  values  which  is  both 
necessary  and  sufficient  for  the  desired  measurement  objective.  How¬ 
ever,  such  a  mathematically  precise  approach  is  not  generally  feasible 
due  to  imprecise  descriptions,  and  the  constraints  and  trade-offs  in¬ 
volved  in  the  design  of  the  set  of  measurements.  These  constraints  in¬ 
volve  the  costs  of  implementing  the  measurements,  including  the  man¬ 
power  effort  and  the  usage  of  system  resources ,  as  well  as  the  allow¬ 
able  degradation  of  the  system  performance  due  to  the  presence  of  the 
measurement  facilities.  The  trade-offs  involve  r.any  of  these  same 
factors,  but  consider  the  possible  alternatives  with  the  constraint 
boundaries.  We  will  consider  such  trade-offs  in  the  design  of  the  ARPA 
network  measurements  in  subsequent  sections  of  this  study. 

2.3  Identification  of  the  Measurement  Needs 

The  network  measurement  efforts  have  been  directed  towards 
serving  a  variety  of  measurement  needs.  In  one  aspect,  the  entire  net¬ 
work  is  considered  to  be  an  experiment,  and  the  measurements  are  to 
evaluate  its  usage  and  degree  of  success.  In  a  second  role,  the  meas¬ 
urements  are  to  determine  the  performance  level  of  the  existing  network 
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and  to  ensure  that  it  meets  the  original  design  goals,  while  another 
aspect  is  to  determine  the  areas  in  which  these  goals  should  be  changed. 
In  this  effort,  the  measurement  functions  are  closely  related  to  the 
analytic  and  simulation  modeling  work,  since  each  can  complement  the 
other  in  gaining  insight  into  network  behavior  and  in  developing  new 
techniques  of  network  flew  control  to  improve  performance. 

Since  the  delay  which  a  message  encounters  in  going  through 
the  network  is  a  primary  performance  measure,  we  cure  particularly  con¬ 
cerned  with  delay-related  measurements.  These  include  measurements  of 
the  overall  end-to-end  message  delays,  the  oerrponents  of  the  delay  due 
to  serial  transmission,  queueing,  retransmission,  etc.,  and  the  effect 
of  system  parameters  such  as  channel  bandwidth,  priority  classifica¬ 
tion,  and  packet  lengths  on  the  message  delay.  We  are  also  concerned 
with  measurements  of  the  message  length  distributions  and  the  site-to- 
site  traffic  requirements  since  these  data  are  needed  for  analytic  and 
simulation  modelinq  of  delay-related  network  functions  such  as  routing, 
channel  capacity,  and  network  topology  designs.  Other  measurements 
which  will  provide  needed  data  for  such  designs  include  time-of-day 
load  variations,  the  peak  buffer  storage  requirements,  the  allocation 
of  buffers  between  store-and-forward  and  reassembly  functions,  and  the 
routing  effectiveness  in  avoiding  loops  and  in  bifurcating  flew  in 
heavy  traffic  conditions. 

The  above  cement  of  using  message  delay  as  the  primary  per¬ 
formance  measure  needs  further  clarification  since  the  message  delay  is 
actually  a  random  variable.  Therefore,  we  must  utilize  seme  attribute 
of  the  underlying  distribution  as  the  performance  measure  if  we  are  to 
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have  meaningful  and  quantitative  measure  of  performance.  The  use  of 
the  average  delay  is  most  cannon  in  the  literature,  but  is  not  neces¬ 
sarily  sufficient  for  the  total  design.  For  example,  we  are  also  in¬ 
terested  in  some  measure  of  the  variation  in  the  delay  (such  as  the 
variance  or  the  90th  percentile)  and  therefore,  need  more  than  just 
average  delay  information.  In  many  cases,  we  shall  atterpt  to  measure 
actual  distributions  to  obtain  this  additional  information,  especially 
since  such  information  is  of  value  in  the  modeling  efforts  as  well. 
These  measurements  should  include  distributions  of  message  lengths, 
packet  lengths,  queueing  delays,  buffer  requirements,  and  of  course, 
the  message  delay  itself. 

Many  of  these  measurements  are  associated  with  the  queueing 
phenomena  since  it  can  cause  a  significant  portion  of  the  total  delay, 
and  can  be  modified  by  appropriate  strategies  in  selecting  the  queue 
and  service  disciplines.  Such  changes  are  less  expensive  than  brute 
force  increases  in  oomnunicaticn  line  bandwidth  or  processor  speed,  and 
may  accomplish  similar  reductions  in  the  effective  message  delays. 

There  are  several  ways  of  measuring  queueing  congestion  including 
measurements  of  the  time  in  the  queueing  system,  the  number  in  the  sys¬ 
tem,  and  the  busy  period  of  the  server.  The  first  two  factors  are  of 
concern  to  us  since  they  directly  relate  to  the  message  delays  and  the 
buffer  requirements,  and  will  therefore  be  part  of  our  measurement  re¬ 
quirements.  The  busy  period  is  difficult  to  interpret  in  such  a  con¬ 
text,  so  it  was  not  considered  to  be  of  particular  interest. 

The  desire  to  utilize  the  measurement  facilities  and  data  to 
improve  the  network  design  can  be  considered  to  be  a  feedback  process 
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in  which  design  deficiencies  are  detected  arri  subsequently  corrected. 
Such  deficiencies  would  not  necessarily  mean  that  the  original  design 
was  poorly  done,  but  could  be  due  bo  changing  demands  on  the  network, 
or  in  other  cases  could  be  a  matter  of  fine-tuning  the  network  param¬ 
eters.*  Of  course,  gross- tuning  would  also  be  acccrplished  if  im¬ 
balances  existed  in  the  overall  system,  e.g.,  if  the  s tor e-and- forward 
processor  could  not  support  the  transmission  speeds  of  the  communica¬ 
tion  lines.  The  occurrence  of  such  '’bottlenecks"  will  be  the  subject 
of  certain  exploratory  measurements  which  will  search  for  exceptional 
conditions,  in  addition  to  gathering  information  about  the  overall 
network  behavior. 

2.4  Hie  Design  of  a  Set  of  Network  Measurements 

Hie  general  design  goal  of  the  network  measurement  effort 
was  to  provide  a  flexible  set  of  measurement  tools  which  would  allow 
an  experimenter  to  evaluate  various  aspects  of  the  network  behavior, 
without  having  to  solicit  the  assistance  of  other  network  sites,  and 
without  introducing  an  excessive  burden  on  the  IMP’S  and  ccrmunicacion 
facilities.  These  measurements  were  to  be  designed  to  support  the  pre¬ 
viously  described  needs  of  network  performance  evaluation  and  providing 
insight  and  feedback  for  the  development  of  analytic  arri  simulation 
models  of  network  behavior,  and  several  of  these  design  considerations 
are  discussed  in  the  following  paragraphs. 

ir 

Parameters  which  vary  with  loading  might  be  made  self-adaptive  by 
introducing  a  continual  measurement  and  feedback  loop  in  the  design. 
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a.  A  priori  expectations:  Due  to  the  fact  that  the  network 
consists  primarily  of  time-shared  computer  facilities,  we  took  advan¬ 
tage  of  previous  measurement  results  (regarding  terminal- to- computer 
message  lengths  aid  response  times)  in  the  design  of  the  data  gathering 
techniques . 

b.  Time  and  space  distribution  of  measurements;  A  spectrum 
of  measurement  techniques  is  possible  ranging  from  taking  very  detailed 
measurements  at  a  single  node,  to  taking  more  gross  measurements  across 
the  entire  network.  The  measurement  facilities  of  the  ARPA  network 
cover  a  large  portion  of  this  spectrum  by  means  of  selective  control 
over  the  data  gathering  routines  which  are  enabled  at  the  various  IMP'S, 
but  are  limited  by  several  factors  including  (1)  the  rate  at  which  the 
destination  HOST  can  accept  the  data,  (2)  the  bandwidth  required  for 
the  data  transmission,  and  (3)  the  resulting  bias  in  the  measurement 
data  due  to  the  use  of  the  network  for  transmission. 

Our  experience  has  indicated  that  extensive  data  can  be 
accumulated  simultaneously  from  our  three  nearest  neighbor  IMP's,  or 
from  an  arbitrary  pair  of  IMP'S,  but  that  severe  performance  degrada¬ 
tion  occurs  when  eight  or  ten  IMP's  are  monitored  simultaneously.  Of 
course,  these  figures  should  be  considered  to  be  guidelines  rather  than 
well  defined  limits,  since  the  measurement  effects  are  quite  dependent 
upon  the  type  of  measurements  being  made  and  the  other  network  utiliza¬ 
tion  at  the  time. 

c.  Hardware  versus  software  measurement  techniques?  Hardware 
measurement  techniques  have  the  advantage  of  introducing  little  or  no 
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degradation  to  the  system  being  monitored.  However,  the  techniques 
have  limited  versatility  and  do  not  have  ready  access  to  data  that  are 
kept  in  software  formats.  For  example,  packet  lengths  can  be  deter¬ 
mined  fran  a  pair  of  buffer  pointers,  but  these  pointers  are  kept  in 
core  locations,  and  therefore,  the  hardware  probes  oould  not  assess  the 
values  without  the  aid  of  special  software  support.  The  need  for  such 
software  support  in  either  case,  as  well  as  the  cost  and  inflexibility 
of  the  hardware  monitoring  techniques,  led  to  the  adoption  of  an  entire 
software  monitoring  approach. 

d.  Transmission  of  the  measurement  data;  Ihe  data  must  be 
transmitted  to  the  experimenter  for  analysis,  and  preferably  this 
transmission  should  be  in  a  near  real-time  manner  to  allow  interactive 
experimentation .  Hie  logical  choice  of  transmission  media  is  to  use 
the  network  itself,  and  indeed,  this  was  the  selected  mode  of  operation, 
although  it  introduced  artifact  considerations  which  will  be  discussed 
later.  The  alternative  of  storing  the  data  for  transmission  during  off 
hours  would  require  a  bulk  store  facility  at  each  IMP,  or  HOST  assist¬ 
ance  in  such  storage,  and  neither  was  felt  to  be  a  desirable  alterna¬ 
tive.  However,  the  interactive  experimentation  which  one  can  perform 
with  a  real-time  feedback  of  the  resulting  data  has  been  found  to  be 
extremely  beneficial,  if  not  absolutely  necessary,  and  other  alterna¬ 
tives  which  do  not  provide  such  feedback  are  of  questionable  utility. 

e.  Artifact  considerations:  Transmitting  the  measurement 
data  in  the  regular  message  stream  introduces  perturbations  in  several 
portions  of  the  data.  Any  message  statistics,  such  as  length 
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distributions  and  site-to-site  traffic  are  biased  due  to  these  data 
messages,  as  are  the  statitstics  on  round  trip  delays,  buffer  utiliza¬ 
tion,  and  modem  channel  traffic.  A  more  subtle  artifact  is  introduced 
in  the  input  handling  characteristics  of  the  IMP,  since  any  full  buffer 
must  be  serviced  and  an  empty  buffer  linked  in  its  place  before  the 
next  message  starts  to  arrive  on  that  channel.  This  condition  can 
occur  simultaneously  on  all  input  channels,  so  the  code  involved  in  the 
input  servicing  must  be  kept  to  a  minimum.  For  this  reason,  statistics 
are  taken  at  the  modan  outputs  rather  than  the  inputs  whenever  possible 

A  third  form  of  artifact  of  concern  is  the  creation  of  dis¬ 
ruptive  conditions  in  a  situation  when  they  would  not  otherwise  occur. 
For  example,  if  statistics  messages  create  a  shortage  of  buffers  at  the 
Network  Measurement  Center  IMP,  that  IMP  be  cores  blocked  for  subsequent 
messages  to  its  HOST.  Neighboring  IMP'S  may  then  have  their  store-and- 
forward  buffer  supply  exhausted  as  they  fail  to  receive  acknowledge¬ 
ments  on  packets  sent  to  the  NMO.*  Fortunately,  this  condition  does 
not  persist  due  to  time-out  periods  which  allow  such  messages  to  be 
discarded,  but  the  transient  effect  could  easily  occur  if  the  experi¬ 
ments  are  not  carefully  designed  to  avoid  such  conditions.  These  and 
other  artifact  considerations  will  be  discussed  in  more  detail  in 
Section  3.4. 

f .  Data  ocrtpaction  and  selection  techniques:  The  concern 
for  minimizing  the  data  transnission  requirements  for  statistics 

* 

The  analysis  of  such  blocking  and  congestion  effects  has  been  con¬ 
sidered  as  another  part  of  the  UCLA  network  research,  and  has  been 
reported  by  Zeigler  and  Kleinrock  (ZE71) . 
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messages  leads  one  to  utilize  data  ccnpacting  techniques  such  as  total 
counts,  averages,  histograms,  etc.  instead  of  transmitting  the  raw  data 
samples.  The  need  for  processing  in  such  data  ccrpaction  ccherres  re¬ 
sults  in  a  trade-off  between  data  compression  and  data  transmission, 
with  the  optimal  balance  of  the  two  being  determined  by  their  costs  and 
the  value  of  the  resulting  data  to  the  experimenter.  The  problem  is 
ocrplicated  by  the  fact  that  the  measurement  tools  are  designed  for  an 
open  set  of  experiments,  such  that  the  designer  must  make  decisions 
based  on  an  inoarplete  knowledge  of  the  total  requirements.  No  guide¬ 
lines  can  be  given  for  these  decisions  other  than  cannon  sense,  intui¬ 
tion,  and  close  coordination  with  the  expected  users  of  the  measurement 
facilities.  In  the  ARPA  network  measurements,  we  were  only  concerned 
with  determining  the  general  shapes  of  density  functions,  and  with 
estimates  of  average  values  to  the  order  of  ±  10%.  However,  these  same 
facilities  did  allow  us  to  precisely  verify  the  rreasurenent  capabilities 
on  known  traffic  experiments,  due  to  the  ocmbination  of  histograms, 
total  word  counts,  etc. 

Our  primary  application  of  data  compacting  techniques,  in  the 
sense  of  reducing  a  set  of  data  values  to  some  considerably  smaller  set 
of  numbers,  was  in  the  usage  of  histograms  for  message  and  packet 
lengths,  and  total  counts  of  the  vrords  transmitted,  the  number  of  ac¬ 
knowledgements ,  RPNM's,  retransmissions,  etc.,  and  averages  of  round 
trip  times.  However,  an  even  more  important  aspect  of  minimizing  the 
data  transmission  is  the  careful  selection  of  what  data  is  to  be  col¬ 
lected.  These  considerations  enter  into  both  the  design  of  the  meas¬ 
urement  package  itself,  and  the  design  of  the  individual  experiments. 
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Our  measurement  tools  were  separated  into  several  types,  each  of  which 
can  be  selectively  enabled  or  disabled  at  any  IMP.  Therfore,  only  a 
subset  of  the  measurement  facilities  would  be  enabled  at  a  selected  set 
of  nodes,  resulting  in  a  considerable  reduction  in  the  data  transmis¬ 
sion  artifact  and  also  minimizing  the  data  retrieval  and  reduction 
problem  at  the  NMD. 

An  additional  technique  to  minimize  the  data  transmission 
requirement  is  bo  use  a  "report  by  exception"  method.  This  technique 
results  in  data  being  sent  only  when  a  certain  change  has  been  noted 
frcm  seme  stored  set  of  values,  and  in  one  sense,  only  sends  data  wfr'n 
something  interesting  has  happened.  The  method  requires  additional 
storage  for  the  set  of  caparison  values,  and  additional  processing  for 
the  caparison  operations,  but  of  more  concern,  it  requires  a  precise 
definition  of  what  is  of  interest,  and  may  result  in  a  huge  number  of 
"exception  reports"  when  unusual  ciramstanoes  such  as  line  errors  or 
buffer  congestion  occur.  These  are  situations  in  which  one  certainly 
does  not  want  to  send  a  multitude  of  statistics  messages,  and  although 
logic  could  be  built  in  to  send  no  more  often  than  every  N  seconds,  the 
added  corrplexity  detracts  frcm  the  desirability  of  the  method.  Our 
usage  of  the  technique  was  limited  to  reporting  arrival  times  (only 
sent  when  arrivals  occur)  and  traces  (only  sent  when  certain  flags  are 
set) ,  but  in  general  the  measur orient  philosophy  was  more  one  of  "take 
the  data  every  N  seconds,  and  discard  it  at  the  NMD  if  it  is  not  of 
interest".  This  approach  also  has  the  advantage  of  placing  the  deter¬ 
mination  of  what  is  "of  interest"  in  a  local  facility  where  it  can  be 
modified  depending  cn  the  particular  experiment  being  conducted. 
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g.  Data  reduction  and  analysis:  The  design  of  the  data 
reduction  and  arialysis  methods  sou Id  be  considered  in  an  overall  design 
to  avoid  inocmpatabilities  with  the  data  acquisition  methods.  In  the 
case  of  the  ARPA  network  statistics  messages,  each  message  has  tag  in¬ 
formation  which  identifies  its  source,  its  type  of  measurement  data  and 
the  IMP  clock  value.  Information  can  be  retrieved  by  selecting  mes¬ 
sages  based  on  these  tag  fields  and  then  indexing  to  the  appropriate 
data  field  within  the  message.  Our  present  technique  is  to  format  the 
desired  information,  and  print  it  out  with  suitable  annotation  to  pro¬ 
vide  a  readable  summary  of  the  measurement  data.  Most  extensive  data 
reduction  and  analysis  techniques  could  be  utilized,  particularly  when 
such  resources  become  available  via  the  network. 

2.5  The  Set  of  Measurement  Tools 

The  measurement  facilities  can  be  categorized  into  three 
classes;  (1)  those  within  the  IMP,  (2)  those  that  have  been  developed 
at  the  NMC  (UCIA) ,  and  (3)  those  available  in  cooperating  HOST  sites. 

The  IMP  generated  statistics  consist  of  accumulated  statistics,  snap¬ 
shots,  trace  data,  arrival  times,  and  status  reports  and  the  IMP  also 
has  the  capability  of  artificial  traffic  generation.  The  NMD  facilities 
include  a  more  extensive  artificial  traffic  generation  capability, 
having  either  deterministic  or  pseudo-random  message  lengths  and  inter- 
nessage  times,  as  well  as  the  ability  to  enable  to  disable  the  various 
DIP  statistics  generators,  to  scan  the  network  for  interesting  traffic 
activity,  and  to  perform  reduction  and  print-cut  of  annotated  data 
summaries.  The  cooperating  HOST  facilities  are  primarily  a  future 
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expectation  rather  than  a  present  capability,  but  some  cooperative  work 
has  been  done,  particularly  with  SRI.  Future  cooperative  experiments 
may  include  the  measurement  of  HOST  induced  delays  and  total  user 
oormand-to-response  round  trip  d  3 lays  as  well  as  the  usage  of  remote 
site  data  reduction  and  display  techniques  for  data  analysis. 

Each  of  the  measurement  facilities  is  described  in  seme  de¬ 
tail  in  the  foilwing  sections  starting  with  the  IMP  generated  statis¬ 
tics.  The  IMP  generates  these  data  messages  in  a  set  of  background 
(low  priority)  prograns  which  are  selectively  enabled  as  needed.  The 
statistics  are  sent  to  the  NfC  at  periodic  time  intervals  from  the 
"fake  HOST"  nechanism  in  the  IMP'S,  although  being  lew  priority  pro¬ 
grams,  their  generation  can  be  delayed  considerably  if  the  IMP  beocmes 
congested.  The  individual  IMP  statistics  programs  are  sutmarized  in 
Sections  2.5.1  to  2.5.6,  and  the  reader  is  referred  to  the  BEN  descrip¬ 
tion  of  their  mechanization  if  more  detail  is  desired.* 

2.5.1  Accumulated  Statistics 

The  accumulated  statistics  routine  has  been  utilized  more 
frequently  than  any  of  the  other  measurement  tools,  since  it  provides  a 
sutmary  report  of  the  activity  at  each  node  where  such  statistics  have 
been  enabled.  These  data  reports  include  histograms  of  the  lengths  of 
any  HOST- to- IMP  or  IMP-to-HOST  messages,  and  of  packets  which  were 
transmitted  on  the  modem  output  lines,  as  well  as  counts  of  ACK's, 
RFTW's,  input  errors,  retransnissions ,  total  words  sent,  and  other  data 

*BBN  Report  No.  1822,  "Specification  for  the  Connection  of  a  HOST  and 
an  IMP,  "Bolt,  Beranek  and  Newman,  Inc.,  Cambridge,  Mass.,  Feb.  1970. 
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of  concern.  Each  such  data  message  represents  the  activity  over  a 
12.8  second  interval,  this  period  being  determined  primarily  by  the 
concern  for  counter  overflow.  The  data  is  sent  to  the  Network  Measure¬ 
ment  Center  as  a  390  byte  message,  which  is  then  inspected  and  either 
discarded,  printed,  or  put  in  a  file  for  further  data  ccrpacting.  When 
printed,  the  data  is  annotated,  reformatted,  and  in  some  instances  ad¬ 
ditional  values  are  oonputed,  with  a  resulting  print-out  as  shewn  in 
Figure  2.5.1.  The  various  fields  of  this  data  format  are  described 
below. 

Part  1  of  the  data  gives  message  size  statistics  for  both 
HOST- to- IMP  and  IMP-to-HOST  data  transfers.  (In  cases  where  more  than 
one  HOST  share  the  same  IMP,  these  data  are  the  aenposite  of  all  such 
transfers.)  The  statistics  on  single  packet  messages  are  accumulated 
in  a  logarithmic  scale  histogram,  while  multipacket  traffic  is  recorded 
as  a  uniform  interval  histogram.  The  average  nurber  of  words  in  the 
last  packet  of  a  message  is  also  given,  which  in  many  cases,  allows  one 
to  more  precisely  compute  the  actual  message  lengths.  In  the  example 
shown,  there  were  a  total  of  220  messages  received  from  the  HOSTS,  of 
which  208  were  single  packet  messages  and  twelve  were  two  packets  in 
length.  A  single  message  was  sent  to  the  HOST,  namely  this  statistics 
message  which  can  be  seen  to  consist  of  four  packets  v/ith  only  six 
words  in  the  fourth  packet  for  a  total  message  length  of : 

length  =  3  •  63  +  6  =  195  IMP  words. 

The  second  part  of  Figure  2.5.1  shews  that  these  220  messages 
were  split  fairly  evenly  between  six  destinations.  These  sites  were 
st  lected  as  destinations  of  the  artificial  traffic  to  shew  the 
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Figure  2.5.1,  A  Typical  Accumulated 
Statistic  Print-Cut 
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Figure  2.5.1,  Continued 
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difference  in  delays  when  messages  are  sent  through  several  store-and- 
forward  nodes.  The  locatic-is  of  these  sites  relative  to  UCLA  are  shown 
in  Figure  1.3.1,  and  the  round  trip  times  can  be  reasonably  well  cor¬ 
related  with  the  topological  locations,  even  though  subsequent  investi¬ 
gation  showed  that  the  packet  routing  was  not  as  expected. 

The  round  trip  times  are  measured  frcm  the  receipt  of  the 
first  packet  of  the  message  at  the  originating  IMP,  until  the  receipt 
of  the  RFNM  by  this  IMP.  (The  RINM  is  generated  by  the  destination  IMP 
when  the  first  packet  has  been  accepted  by  the  HOST.)  The  data  values 
are  accunulated  as  a  sun  of  round  trip  times  and  a  count  of  the  number 
of  round  trips,  with  the  average  being  ccmouted  at  the  M'C. 

The  selection  of  the  clock  resolution  for  the  round  trip 
times  was  found  to  be  quite  important.  Initially  the  25.6  msec.  IMP 
clock  was  utilized,  but  this  was  too  coarse  since  many  round  trip  times 
are  of  the  order  of  20  msec.  The  alternative  100  usee,  clock  was  then 
utilized,  and  while  the  resulting  resolution  was  very  good,  the  16-bit 
total  count  frequently  overflowed .  On  sore  tests,  this  overflew  oould 
be  detected  and  corrected  by  reasonableness  checks,  but  for  experiments 
involving  hundreds  of  round  trips,  such  correction  was  not  possible. 

The  clock  resolution  was  then  changed  to  0.8  msec,  by  an  IMP  code 
change  which  shifted  the  100  usee,  clock  value  by  three  bit  positions 
prior  to  recording  the  time.  The  0.8  msec,  resolution  is  quite  adequate 
for  the  round  trip  times,  and  avoids  any  overflow  until  a  total  round 
trip  time  of  52.5  seaonds  has  been  exceeded.  Even  this  range  is  sub¬ 
ject  to  overflow  in  sane  cases,  e.g.,  for  1000  round  trips  with  an 
average  round  trip  time  of  more  than  52.5  msec.,  but  such  overflow 
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should  be  correctable  in  most  cases. 

Part  3  lists  the  activity  for  each  HOST,  both  real  and  fake, 
the  latter  being  referred  to  as  GHDST's.  The  table  helps  to  determine 
which  HOST'S  contributed  to  the  composite  HOST-to-IMP  behavior  as  re¬ 
corded  in  Parts  1  and  2.  The  "fake  HOST'S"  are  those  listed  previously 
in  Section  1.3. 

The  activity  on  the  individual  modem  channels  is  reoordec  in 
Part  4.  HELLO  and  IHY  (I  Heard  You)  messages  are  sent  periodically, 
and  proper  counts  indicate  that  the  lines  are  active.  (Hie  HELIX)  mes¬ 
sage  is  cartoined  with  the  routing  table  update  message.)  The  "#PKTS 
RBCVD"  total  includes  these  HELDO  messages,  and  any  ACK's  and  RFNM's 
that  have  been  received,  as  well  as  the  nimber  of  actual  message 
packets  that  have  been  received  (both  those  received  successfully  and 
any  having  check  sun  errors.)  Other  data  include  the  number  of  RFNM's 
sent,  the  total  number  of  data  words  sent,  the  number  of  times  an 
arrival  found  the  free  storage  list  empty,  and  in  such  cases,  the  num¬ 
ber  of  times  that  an  unacknowledged  store-and-forward  packet  was  dis¬ 
carded  to  free  an  input  buffer.  Each  of  these  items  are  recorded 
separately  for  the  various  modem  channels  connected  to  the  IMP,  except 
for  the  number  of  retransmissions.  No  information  is  kept  as  to  the 
channel  on  which  a  packet  was  originally  sent,  so  when  retransmissions 
are  required,  only  a  composite  total  is  kept. 

If  we  consider  the  data  from  channel  1  (to  SRI)  we  see  that  a 
total  of  335  packets  were  received,  consisting  of  the  23  IHY's,  156 
RFWl's  (for  which  the  156  ACK's  were  sent),  and  156  ACK's  (frcm  the 
packets  sent  via  SRI) .  These  values  agree  within  one  count  to  that  of 
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Part  5,  but  the  data  for  channel  3  (to  RAND)  do  not,  because  of  the  65 
ACK's  sent,  when  about  40  would  have  been  expected.  This  effect  can 
not  be  resolved  from  the  accumulated  data  at  one  site  due  to  the  lack 
of  additional  input  data.  Such  data  is  not  available  because  of  the 
input  time  constraints  described  in  Section  2.4  (e) . 

The  final  section  of  Figure  2.5.1  shews  logarithmic  scale 
histograms  of  packet  lengths  on  each  modem  channel.  Only  the  message 
content  of  the  transmissions  are  recorded  in  these  histograms,  i.e., 
they  do  not  include  the  HELLO,  IHY,  ACK,  or  RFNM  traffic.  The  choice 
of  the  logarithmic  scale  was  due  to  the  a  priori  expectations  on  the 
length  distribution  and  will  be  discussed  in  detail  in  Chapter  3. 

The  various  portions  of  the  accumulated  data  can  be  utilized 
together  to  develop  a  reasonably  good  picture  of  the  activity  over  the 
12.8  second  interval.  In  the  example  of  Figure  2.5.1  we  know  the 
total  number  of  messages  from  the  HDST,  their  final  destinations,  and 
their  average  round  trip  times.  The  HDST-IMP  histograms  give  us  in¬ 
formation  of  the  message  size  distribution,  and  in  the  modem  channel 
histograms  give  similar  data  for  the  packet  size  distribution  as  well 
as  channel  utilization  information.  In  this  example,  the  modem  channel 
statistics  shew  that  more  packets  were  sent  via  SRI  than  was  expected 
from  the  routing  table  at  the  UCLA  IMP,  which  would  normally  send 
about  half  of  these  packets  via  RAND.  Such  data  can  lead  the  experi¬ 
menter  to  probe  further  into  the  cause  of  such  behavior,  but  to  do  so, 
he  needs  the  routing  table  information  which  is  contained  in  another 
set  of  measurements  called  snap-shots. 
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2.5.2 


Snap-Shot  Statistics 


Each  snap-shot  statistics  message  contains  the  queue  lengths 
and  the  routing  table  information  for  that  particular  IMP  and  that  par¬ 
ticular  instant  of  time.  These  values  are  recorded  and  sent  at  0.82 
second  intervals,  this  time  period  being  a  reasonable  oorrpromise  be¬ 
tween  the  desire  to  see  a  time-sequence  of  state  changes,  and  the  con¬ 
flicting  desire  to  reduce  the  artifact  caused  by  sending  the  statistics 
too  frequently.  Like  all  of  the  other  measurement  tools,  the  snap¬ 
shots  can  be  enabled  or  disabled  at  each  individual  IMP. 

Before  describing  the  snap-shot  measurements  of  the  queue 
lengths,  some  background  is  needed  on  the  internal  operation  of  the  IMP 
and  the  interconnections  of  the  various  queues  within  the  IMP.  Figure 
2.5.2  shows  a  simplified  picture  of  this  queue  structure,  which  in¬ 
cludes  the  high  priority  task  queues,*  and  the  hierarchy  of  queues  for 
the  modem  and  HOST  output  channels.  Each  modem  output  channel  also 
has  a  SEOT  queue  which  holds  the  packets  which  have  been  transmitted, 
but  not  yet  acknowledged.  When  an  ACK  is  received,  the  appropriate 
packet  buffer  is  returned  to  the  free  storage  list,  but  if  no  ACK  is 
received  by  the  end  of  the  time-out  period  (of  about  ion  msec.),  the 
packet  is  retransmitted. 

The  buffer  handling  statistics  are  divided  into  two  classes; 
(1)  store-and- forward  buffers  for  the  modem  output  functions,  and  (2) 

The  task  queue  is  a  linked  list  of  service  requests  for  routing  of 
packets,  routing  updates,  acknowledgement  handling,  etc.  Its  priority 
is  such  that  the  background  snap-shot  routine  will  be  deferred  any 
time  that  a  task  requires  service,  and  therefore  it  is  not  measurable 
by  this  method. 
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Figure  2.5.2.  A  Simplified  Model  of  the  IMP  Queue  Structure 


re  as  sent*  ly  buffers  for  the  messages  which  are  to  be  sent  to  the  local 
HOST.  Upper  limits  are  set  on  each  of  these  two  functions,  and  thc^r 
total  is  presently  limited  to  41  buffers.* 

A  typical  formatted  and  annotated  snap-shot  print-out  is 
shown  in  Figure  2.5.3,  with  this  particular  message  being  the  state  of 
the  SRI  IMI  during  a  test  in  which  a  moderately  heavy  traffic  load  was 
being  sustained  between  UCLA  and  Utah.  Part  4  of  the  data  shows  that 
six  store- and- forward  buffers  are  in  use,  and  the  utilization  of  these 
buffers  is  shown  in  Part  2,  where  one  buffer  is  on  the  regular  output 
queue  to  Utah,  another  is  on  the  priority  queue,  three  are  on  the  SENT 
queues,  and  the  sixth  buffer  is  the  one  being  transnitted.^ 

The  third  part  of  the  print-out  shews  the  routing  table  at 
the  instant  the  snap-shot  was  taken,  which  includes;  (1)  the  HOP#,  i.e., 
the  nurrber  of  store-and-forward  "links"  to  get  to  the  various  destina¬ 
tions,  (2)  the  estimated  delay  to  get  to  the  other  sites  (in  arbitrary 
units  of  delay) ,  (3)  the  best  modem  channel  to  take  to  get  to  the  vari¬ 
ous  sites,  and  (4)  the  status  of  each  HOST.  It  should  be  noted  that 
although  the  routing  tables  are  based  on  the  output  queue  lengths,  the 
routing  updates  and  snap-shots  occur  at  different  times,  so  one  can  not 
necessarily  correlate  the  two  sets  of  values. 

1 - 

There  are  actually  a  total  of  about  70  buffers  in  the  IMP,  but  several 
are  dedicated  to  high  priority  functions  such  as  input  handling,  and 
others  form  a  contingency  pool  to  avoid  deadlock  conditions.  More 
detailed  information  on  this  and  other  IMP  functions  can  be  found  in 
Reference  HE70. 

^These  two  accountings  of  the  buffers  do  not  always  agree  since  the 
snap-shot  data  values  are  gathered  as  a  low  priority  background 
routine,  and  therefore  may  be  preempted.  Such  preemption  can  result 
in  differing  values  when  the  routine  is  continued  at  a  later  time. 
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Figure  2.5.3,  A  Typical  Snap-Shot  Print-Out 
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The  snap-shot  measurements  were  developed  primarily  with 
routing  measurements  in  mind,  but  have  also  been  found  to  be  useful  for 
other  experiments  involving  the  queueing  behavior  of  the  IMP'S.  Some 
examples  of  these  experiments  will  be  given  in  Chapters  4  and  5. 

2.5.3  Trace  Statistics 

The  trace  capability  allows  one  to  literally  follow  a  message 
through  the  network,  and  to  learn  of  the  route  which  it  takes  and  the 
delays  which  it  encounters.  However,  it  is  one  of  the  more  complicated 
of  the  measurement  data  formats  to  describe  because  the  interpretation 
of  the  data  depends  on  the  function  of  the  IMP  handling  the  message, 
i.e.,  either  source,  s tor e-and- forward ,  or  destination,  as  well  as 
depending  on  the  possibility  of  retransmissions  or  multiple  copies  of 
packets . 

A  typical  store  and  forward  situation  will,  when  traced, 
result  in  a  record  of  (1)  the  dme  of  arrival,  (2)  the  time  at  which 
the  message  is  processed,  i.e.,  put  on  an  output  queue,  (3)  the  time  at 
which  transmission  is  initiated,  and  (4)  the  time  when  the  ACK  is  re¬ 
ceived.  In  the  case  of  a  retransmission,  this  latter  time  is  the  time 
of  retransmission,  and  is  so  indicated  by  a  tag  bit  in  the  data  block. 
At  the  destination  IMP,  this  fourth  word  is  the  time  at  which  the  IMP- 
to-HOST  transnission  was  completed.  All  times  are  recorded  in  terms  of 
a  clock  with  a  100  usee,  resolution,  and  with  a  full  scale  range  of 
about  6.5  seconds  (for  a  16-bit  word). 

The  other  information  in  the  trace  data  message  includes  the 
output  channel  (if  on  a  modan  channel) ,  the  HOST  number,  or  the  "fake 
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HOST"  number  plus  three,  as  well  as  the  entire  header  of  the  packet. 

The  header  contains  the  source,  the  destination,  the  link  number,  the 
message  number,  the  packet  number,  and  the  priority/non-priority  status. 

A  typical  trace  data  message  is  shewn  in  Figure  2.5.4,  which 
records  the  times  of  the  various  events  at  the  SDC  IMP,  which  also  hao- 
pens  to  the  destination  of  the  message.  The  message  was  four  packets 
in  length,  and  was  sent  frem  HOST  #0  at  UOA  on  link  number  one.  (The 
GHOST  3  destination  is  the  "fake  HOST'  which  is  used  to  discard  arti¬ 
ficial  traffic.  Such  discarding  is  neoessary  to  generate  the  FFNM  for 
the  logic  link.) 

The  interarrival  times  can  be  obtained  from  the  T(IN)  data, 
and  indicate  the  presence  of  seme  interference  traffic  since  the  packets 
are  otherwise  contiguous  at  23.0  msec,  intervals.  Since  this  data 
message  is  from  the  destination  IMP,  the  values  of  T (QUEUE)  are  the 
times  at  which  the  packets  were  placed  on  the  reassembly  queue,  and  a 
comparison  of  the  clock  readings  shews  that  the  transmission  to  the 
discard  "fake  HOST"  did  not  begin  until  after  the  last  packet  of  the 
message  was  received,  i.e.,  the  entire  message  had  been  reassembled. 

The  value  of  T(*)  is  the  time  at  which  the  transmission  to  the  HOST  was 
completed,  with  the  discard  "fake  HOST"  accepting  a  full  packet  of  data 
in  about  2.5  msec.  Similar  trace  data  could  have  been  obtained  at  each 
IMP  which  handled  the  message,  although  the  data  content  would  have  dif¬ 
fered  in  some  aspects  as  described  earlier. 

An  IMP  will  generate  trace  data  if  it  handles  a  packet  with 
its  trace  bit  set  and  if  the  trace  function  has  been  enabled  at  that 
particular  IMP.  The  trace  bit  in  the  packet  header  can  be  set  in 
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Figure  2.5.4,  A  Sarple  Trace  Print-Out 


either  of  two  ways;  (1)  if  the  trace  bit  was  set  in  the  leader  of  the 
message  as  received  from  the  HOST,  or  (2)  by  means  of  the  autotrace 
feature  of  the  measurements.  The  latter  function  allows  one  to  trace 
every  Nth  message  originating  at  one  or  more  IMP'S. 

It  is  important  to  note  that  the  various  statistics  messages 
are  indeed  "messages",  and  therefore  are  included  in  the  autotrace 
count,  and  will  be  traced  if  they  are  so  chosen.  The  frequent  status 
report  nossages*  have  therefore  lessened  the  value  of  the  autotrace 
feature  for  use  in  monitoring  the  early  network  usage  on  a  "report  by 
exception"  basis.  That  is,  one  could  have  otherwise  obtained  data  on 
network  message  transmissions  without  a  continual  polling  of  the  net¬ 
work  nodes. 

2.5.4  Arrival  Time  Data 

The  times  at  which  packets  arrive  on  a  specified  channel  can 
be  recorded  by  selectively  turning  on  this  measurement  routine.  The 
arrival  times  are  in  terms  of  a  clock  with  100  ysec.  resolution  and 
will  be  sent  to  the  preselected  destination  at  1.6  second  intervals. 
The  allocated  buffering  is  such  that  no  more  than  sixty  such  arrivals 
can  be  recorded  in  the  1.6  second  period,  but  this  is  not  a  problem 
since  the  statistic  of  interest  is  the  interarrival  time,  and  losing 
sere  arrivals  merely  reduces  the  sample  size  semewhat. 


*The  status  report  messages  are  described  in  Section  2.5.5. 
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2.5.5 


Status  Reports 


Reports  are  sent  to  a  specified  HOST  (most  often  the  IMP 
Teletype  at  BEN)  to  indicate  the  status  of  the  IMP's,  the  HOST'S,  and 
the  data  transmission  lines.  These  reports  are  sent  at  52  second 
intervals  if  the  status  has  changed,  or  otherwise  at  14  minute 
intervals . 

2.5.6  Pseudo-Message  Generator 

Each  IMP  can  be  utilized  as  a  source  of  periodic  messages 
referred  to  as  pseudo-messages  since  they  have  no  actual  information 
content.  Such  messages  are  useful  as  artificial  traffic  in  network 
loading  tests,  for  use  as  periodic  traced  messages  for  delay  sampling, 
and  for  any  other  tests  in  which  one  desires  known  selectable  message 
characteristics . 

The  control  parameters  allcw  the  experimenter  to  select  the 
length  of  the  message  fran  0  to  777  16-bit  words,  the  period  at  which 
the  message  is  sent  in  25.6  increments  of  time  (subject  to  having  re¬ 
ceived  any  previous  RFT'JM) ,  the  destination  of  the  messages ,  the  link 
number,  and  to  set  the  trace  bit  if  traces  are  desired.  These  param¬ 
eters  are  single  valued,  e.g.,  the  message  generator  cannot  send  to 
more  than  one  destination  at  a  time  nor  can  it  send  on  more  than  one 
link  at  a  time. 

The  pseudo-messages  originate  from  "fake  HOST"  #3,  and 
typically  are  sent  to  the  discard  "fake  HOST"  at  seme  other  side.  This 
discard  HOST  accepts  messages  at  a  rate  of  about  one  16-bit  word  every 
40  ysec. ,  although  being  a  low  priority  bakeground  routine,  the  discard 
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routine  is  subject  to  preemption,  and  therefore  the  effective  discard 
rate  may  be  considerably  slower  when  the  IMP  is  very  active. 

2.5.7  Network  Measurement  Center  Control  Programs 

Several  routines  have  been  written  at  UCLA  bo  control  the 
data  gathering  and  reduction  processes,  and  have  been  integrated  into 
an  overall  measurement  control  program  of  considerable  flexibility  and 
generality.  This  program  consists  of  routines  which  (1)  accept  meas¬ 
urement  data  from  the  network,  (2)  control  its  processing  and  print 
out,  and  (3)  handle  the  initialization ,  control  and  termination  of 
tests.  The  data  handling  routine  is  a  FORTRAN  program  which  formats 
and  prints  out  the  measurement  statistics  generated  by  the  IMP'S.  The 
program  uses  a  number  of  assembly  language  subroutines  which  interface 
with  the  system  monitor  and  control  the  interrupts,  the  clock  and  the 
IMP  interface. 

The  measurement  program  is  controlled  from  the  operator's 
console  in  a  manner  similar  to  that  used  to  set  up  experiments  at  the 
IMP  console.  For  example,  the  experimenter  would  type  SNAP  (NL 
followed  by  5  (nl)  if  he  wished  to  have  only  every  fifth  snap-shot 
data  message  printed  out.  He  has  similar  control  capabilities  for  the 
accumulated  statistics  and  trace  data  messages ,  as  well  as  having  con¬ 
trol  over  the  destination  of  the  data  (the  line  printer  or  magnetic 
tape) ,  the  source  of  the  data  (frcm  the  IMP  or  magnetic  tape) ,  and 
other  optional  features  such  as  a  decimal  "dunp"  print-out  of  the  re¬ 
ceived  data. 

The  measurement  program  can  be  utilized  in  a  tine-sharing 
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environment,  but  for  extensive  data  gathering  and  traffic  generation 
experiments,  it  is  operated  as  a  dedicated  system  function.  Most  of 
the  tests  which  have  been  run  as  part  of  this  research  have  required 
the  latter  type  of  operation,  although  the  use  of  the  time-sharing  is 
also  a  very  pwoerful  mode  of  operation  since  it  provides  auxiliary 
services  to  the  measurement  activity.  These  capabilities,  both  within 
the  UCIA  HOST  and  at  other  network  facilities,  will  be  utilized  exten¬ 
sively  in  future  network  monitoring  and  data  analysis. 

2.5.8  NM3  Artificial  Traffic  Generation 

One  of  the  routines  which  the  NM2  control  program  can  call 
is  the  artificial  traffic  generator.  This  routine  allows  the  user  to 
specify  which  of  the  63  possible  generators  that  he  wishes  to  utilize, 
and  to  define  the  destination,  the  initial  message  length,  the  final 
message  length,  the  increments  for  successive  runs,  the  duration  of  the 
run  in  milliseconds,  and  the  number  of  messages  sent  on  each  link.  If 
a  non-zero  length  increment  has  been  specified,  the  experiment  will 
terminate  sifter  the  end  message  length  has  been  run,  but  for  a  zero 
increment  value,  it  will  create  continuous  traffic  until  stepped  fran 
the  operator's  console.  Statistics  on  the  artificial  traffic  can  also 
be  printed  out  on  the  console  at  the  operator's  request. 

The  artificial  traffic  can  be  either  deterministic,  (pre¬ 
defined  message  lengths  and  transmission  intervals)  or  can  be  sto¬ 
chastic  with  pseudo-randem  message  lengths  and/or  intertransmission 
times.  In  the  NMC  implementation ,  these  two  variables  are  selected 
from  tables  of  random  digits  having  the  desired  distribution  shape;  and 
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parameters.  (Run-time  generation  of  such  random  variables  was  too  time 
consuming  due  to  the  lack  of  floating  point  hardware  on  the  UCLA 
Sigma  7.) 

Hie  operation  of  the  artificial  traffic  generator  involves 
additional  logic  to  ensure  that  transmission  are  not  attempted  on  any 
link  that  is  blocked,  i.e.,  that  is  waiting  for  a  RFNM  from  a  previous 
transmission.  If  nc  link  is  free  for  use,  the  next  time  interval,  At, 
is  selected  from  the  random  number  table  and  the  program  waits  At 
seconds  to  retry  the  transmission.  A  new  message  length  is  selected 
for  each  transmission  frcm  a  table  of  randan  lengths,  thereby  providing 
the  desired  distribution  of  message  lengths.  Hie  two  tables  are  inde¬ 
pendent  and  can  be  initialized  separately  to  have  different  distribu¬ 
tions  for  the  intertransmission  times  and  message  lengths.  Doth  tables 
are  circular,  i.e. ,  the  pointers  wrap  around  frcm  the  bottom  of  the 
list  back  to  the  top,  but  are  of  differing  lengths  to  avoid  a  cyclic 
repetition  each  time  through  the  lists. 

Artificial  traffic  can  be  sent  to  ary  destination  since  the 
leader  information  can  be  supplied  separately  for  subsets  of  links. 
However,  the  network  traffic  loading  patterns  that  can  be  obtained  by 
such  traffic  are  limited  by  the  topology  of  the  network,  and  by  the 
proximity  of  the  nodes  and  lines  to  the  NMC.  For  example,  the  inter¬ 
arrival  times  of  messages  can  be  reasonably  well  controlled  at  the 
UCLA  IMP  to  observe  the  queueing  and  alternate  routing  behavior,  but 
little  control  can  be  maintained  over  these  factors  at  other  nodes  due 
to  the  approximately  equal  input  and  output  data  rates.  (Alternate 
routing  effects  can  provide  transient  conditions  in  which  the  input 
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and  output  rates  are  not  equal,  but  these  effects  can  not  be  well 
controlled  as  part  of  an  experiment.) 

The  serial  transnission  time  of  the  HOST- ter- IMP  interface 
also  tends  to  limit  the  degree  of  control  over  the  interarrival  times 
of  messages  at  the  IMP,  but  since  the  transnission  rate  is  twice  that 
of  the  IMP- to- IMP  modem  channels,  the  effect  has  not  been  a  serious 
problem  in  the  establishment  of  high  traffic  levels.  Hcwever,  the 
queue  which  may  form  on  the  HOST  side  of  this  interface  will  result  in 
a  tandem  queue  systan  which  is  particularly  difficult  to  analyze.  This 
effect  will  be  discussed  further  in  Section  5.3.1. 

Sinoe  the  modem  channel  requires  about  5  to  20  msec,  to 
service  a  typical  short  message,  the  artificial  traffic  generator  must 
be  able  to  provide  new  arrivals  with  at  least  this  frequency.  The  need 
to  provide  seme  control  over  the  arrivals,  e.g. ,  to  have  approximately 
exponential  interarrival  times,  requires  a  higher  resolution  timer  and 
the  2  msec,  clock  of  the  Sigma  7  was  used  for  this  purpose.  Such  arti¬ 
ficial  traffic  generation  required  the  dedicated  use  of  the  machine, 
since  these  real-time  functions  could  not  be  performed  in  a  time¬ 
sharing  environment. 

2.5.9  HOST  Cooperation  in  Measurements 

Although  a  measurement  ground  rule  was  that  we  would  not 
require  the  assistance  of  the  various  HOST  computers  to  obtain  data, 
HOST  cooperation  can  add  significantly  to  the  confidence  in  the  meas¬ 
urements.  An  early  example  of  such  cooperation  is  described  in  the 
experiment  of  Section  4.2.1  in  which  John  Melvin  of  SRI  modified  his 
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program  to  record  the  console  activity.  The  data  correlated  'quite 
well  with  the  measurement  results.  Further  discussions  of  their  mode 
of  operation  clarified  other  aspects  of  the  data  such  as  the  predomi¬ 
nance  of  5  word  packets  and  5  packet  messages.  Such  cooperation  will 
hopefully  lead  to  more  extensive  measurement  of  HOST  related  network 
activities . 
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CHAPTER  3 


DATA  GATHERING  AND  REDUCTION  TECHNIQUES 

The  network  measurement  package  includes  a  variety  of 
monitoring  techniques  such  as  obtaining  counts ,  averages,  histograms, 
event  times,  and  traces  of  store-and-forward  flow.  The  use  of  a  spec¬ 
trum  of  measurement  methods  was  required  to  meet  the  wide  range  of 
interests  of  the  measurement  project,  including  design  verification 
tests,  performance  evaluation,  support  of  the  analytic  and  simulation 
modeling  efforts,  and  gaining  insights  into  potential  network  improve¬ 
ments.  Some  of  the  results  of  these  tests  and  evaluations  will  be  pre¬ 
sented  in  Chapter  4,  but  first  we  shall  describe  the  techniques  that 
were  developed  for  gathering  and  reducing  the  data. 

3.1  Time  Sequence  Data 

Many  of  the  network  functions  of  interest  can  be  considered 
to  be  stochastic  processes  (i.e.,  time  sequences  of  randan  variables). 
These  functions  include  queueing  processes,  routing  effects,  blocking 
effects,  and  the  random  nature  of  the  user-computer  camuni cations . 

The  time  element  is  an  important  factor  in  each  of  these  areas,  since 
the  sequence  in  which  events  occur  can  alter  the  resulting  behavior  to 
a  considerable  degree.  Therefore  in  many  cases  we  will  be  more  con¬ 
cerned  with  following  the  sequence  of  such  events  rather  than  measuring 
averages  or  distributions. 


Preceding  page  blank 
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3.1.1 


Queue  Lengths  as  a  Function  of  Time 


The  detailed  time  behavior  of  queue  lengths  is  not  of 
particular  concern,  but  we  are  interested  in  a  more  gross  time  scale 
for  the  effects  on  routing  and  possible  blocking  conditions.  Irrieed, 
the  detailed  queue  behavior  is  not  measurable  by  our  methods  due  to  the 
relatively  short  period  in  which  the  queue  lengths  can  change.  A  full 
packet  of  1008  bits  can  be  transmitted  over  the  50  Kbit/sec.  line  in 
about  20  msec. ,  while  our  snap-shot  measurements  of  queue  lengths  occur 
only  every  820  msec,  which  is  as  frequently  as  they  can  occur  without 
introducing  an  excessive  transmission  artifact.  However,  the  snap-shot 
timing  is  satisfactory  for  investigating  longer  term  phenomena  such  as 
congestion  periods  and  their  effect  on  routing  and  blocking,  as  well  as 
the  effectiveness  of  time-outs  and  alternate  routing  in  circumventing 
b.locked  or  defective  lines  or  nodes. 

3.1.2  Routing  Table  Measurements 

Each  IMP  maintains  its  cwn  routing  table  which  it  updates 
about  every  half  second  using  nearest  neighbor  estimates  of  delay  to 
each  destination.  This  procedure  is  adaptive,  although  it  is  not 
optimal  or  even  necessarily  stable.  An  example  of  an  instability  in 
routing  is  the  simple  case  in  which  node  A  routes  messages  to  node  B  to 
be  forwarded  to  their  destination  C,  and  node  B  thinks  that  the  best 
path  to  get  to  C  is  via  node  A.  A  ping-ponging  effect  would  then  occur 
between  A  and  B.  Such  loops  can  occur  as  transient  cases  between  up¬ 
dates,  but  should  not  persist  indefinitely.  Measurements  of  the  IMP 
routing  tables  are  included  in  the  snap-shot  data  which  can  be  taken 
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at  820  msec,  intervals.  The  snap-shot  timing  is  reasonably  synchronized 
across  the  network  to  help  detect  such  loops  and  to  provide  an  instan¬ 
taneous  picture  of  the  entire  net. 

The  delay  estimates  which  are  utilized  in  the  routing  updates 
are  fonred  in  a  matrix  which  includes  all  of  the  destinations  and  all 
of  the  oaiminication  channels  to  the  nearest  neighbors.  For  example, 
the  estimated  delay  to  a  given  uestination,  if  the  channel  to  node  A 
is  taken,  is  the  delay  from  A  to  the  destination,  plus  the  queue  length 
on  the  channel  to  A,  plus  four.  The  delay  is  expressed  in  arbitrary 
units  since  the  intent  is  merely  to  find  the  best  channel  to  take. 

(The  number  four  is  a  threshold  value  for  the  alternate  routing.)  Many 
variations  of  this  routing  update  scheme  are  possible,  and  part  of  the 
measurement  effort  is  to  provide  sane  insight  into  viable  alternative 
procedures  which  may  improve  the  network  performance  (FU71B) . 

3.1.3  Blocking  Effects 

If  a  store-and- forward  node  exhausts  its  buffer  supply ,  it 
becomes  blocked  in  the  sense  that  neighboring  nodes  can  not  success¬ 
fully  transmit  messages  to  it.  This  massive  congestion  can  then  propo- 
gate  throughout  a  surrounding  region  until  such  time  as  the  congestion 
is  cleared  by  successful  transmissions,  alternate  routing,  or  discard¬ 
ing  of  messages  due  to  time  outc .  We  are  interested  in  such  blocking 
effects,  including  the  time  and  place  at  which  they  occur,  the  number 
of  nodes  affected,  and  the  duration  of  the  blockage  (ZE71) .  Unfor¬ 
tunately,  the  present  measurement  data  messages  tend  to  suffer  the  same 
congestion  as  other  messages  in  such  a  blocked  condition,  and  do  not 
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necessarily  reach  the  NMC.  The  fact  that  the  measurements  are  a 
background  task  also  limits  their  effectiveness  in  such  instances  when 
the  IMP  becomes  very  "busy"  with  foreground  tasks.  However,  the  loss 
or  skew  in  the  data  message  timing  is  sane  indication  in  itself  of  the 
blocking  effect,  so  a  first-order  measure  of  the  blocking  can  be 
obtained. 

Blocking  can  occur  in  either  store-and-forward  and/or  re¬ 
assembly  functions  because  each  of  these  functions  has  a  separate  upper 
limit  on  buffer  allocation.  Therefore,  blocking  might  occur  in  the 
reassembly  function,  which  would  not  allow  subsequent  "for  HOST"  in¬ 
puts,  but  the  IMP  could  continue  store-and-forward  message  handling 
including  the  sending  of  snap-shot  data.  The  opposite  effect  could 
also  allow  measurement  data  to  be  obtained,  but  only  if  the  store-and- 
forward  blockage  occurred  at  the  NMC. 

3.1.4  Inferences  of  User  Behavior 

The  observed  network  traffic  can  sometimes  be  used  to  infer 
the  type  of  computer  applications  which  are  being  utilized  via  the 
network.  For  example,  in  one  early  test,  the  traffic  data  clearly 
shewed  a  pattern  of  large  file  transmissions  followed  by  a  period  of 
interactive  traffic,  with  the  interactive  traffic  consisting  of  char¬ 
acter  -by-character  carmands  in  one  direction  and  line-by-line  trai  re¬ 
mission  for  the  replies.  {This  test  is  described  in  some  detail  in 
Section  4.2.1). 

In  other  instances,  the  use  of  the  network  for  interactive 
applications  versus  batch  work  and  file  transfers  should  be  apparent. 
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particularly  when  the  source-destination  pairs  are  considered. 

3.2  The  Use  of  Histograms  to  Estimate  Density  Functions 

A  oonmon  practice  in  the  analysis  of  test  data  is  to  generate 
a  histogram  or  relative  frequency  bar  chart.  Such  a  histogram  gives  an 
estimate  of  the  underlying  probability  density  function,  and  if  the 
sampJe  size  and  the  histogram  interval  size  are  chosen  properly,  the 
histogram  can  model  the  density  function  quite  closely.  Typically  the 
interval  size  is  selected  after  an  inspection  of  the  test  data,  so  that 
the  estimate  of  the  density  function  has  an  acceptable  quantization, 
and  as  many  intervals  would  be  used  as  were  felt  to  be  necessary.  How¬ 
ever,  in  experiments  where  there  is  a  limited  ability  to  store  and/or 
transmit  the  raw  data  values,  sane  aanpaction  of  the  data  is  necessary, 
and  therefore,  one  can  not  use  a  heuristic  aporoach.  Such  an  experi¬ 
ment  would  have  to  be  designed  with  a  predetermined  nurber  of  histo¬ 
gram  intervals  covering  a  particular  range  of  the  variable.  Hie  succe® 
of  such  an  allocation  of  the  histogram  intervals  would  depend  on  the 
amount  of  knowledge  which  the  experimenter  had  about  the  density 
function  being  measured. 

Conventional  histograms  are  composed  of  a  number  of  equal 
size  intervals  covering  the  range  of  the  variable.  Therefore,  for  a 
given  range  and  numbe_  of  intervals,  the  interval  size  is  fixed.  For 
skewed  densities  such  is  the  exponential,  the  resulting  histograms  are 
often  very  poorly  quantized  since  most  of  the  occurrences  fall  in  a 
relatively  small  fraction  of  the  intervals.  A  more  satisfactory  ap¬ 
proach  to  the  measurement  in  such  cases  is  to  use  non-uniform  interval 


65 


sizes,  so  that  the  intervals  are  smaller  in  those  regions  which  are 
most  likely  to  occur.  For  exponentially  skewed  densities,  a  loga¬ 
rithmic  variation  has  been  found  to  be  a  viable  solution  to  this  prob¬ 
lem,  and  the  rest  of  this  section  is  devoted  to  the  analysis  of  such 
logarithmic  histograms. 


3*2.1  The  Rationale  for  the  Use  of  Log- Histograms 

logarithmic  histograms  are  of  particular  interest  because; 

(1)  there  is  an  expectation  that  certain  measurements  will  be  of  an 
exponential  shape,  e.g.,  more  short  packets  than  long  packets,  and  (2) 
the  logarithmic  intervals  are  ocmputationally  easy  to  obtain  in  the 
case  of  base  two  logarithms.  A  binary  computer  data  word  can  be  readily 
categorized  into  one  of  the  logarithmic  intervals  by  merely  finding  the 
most  significant  ONE  in  the  word.  In  this  manner  the  first  interval  is 
for  a  value  equal  to  one,  the  next  for  two  or  three,  the  next  for  four 
through  seven,  etc.  Zero  can  also  be  included  if  it  is  a  meaningful 
value. 

An  intuitive  measure  of  the  capability  of  a  histogram  to 
model  a  density  was  mentioned  in  the  above  example  of  the  exponential 
density  function.  We  concluded  that  non-uniform  interval  sizes  should 
be  used,  because  most  of  the  data  would  otherwise  occur  in  a  few  inter¬ 
vals.  This  observation  leads  to  the  conclusion  that  an  optimal  choiae 
of  non-uniform  interval  sizes  would  be  to  have  an  equal  probability  of 
occurrence  for  each  interval.  Such  a  histogram  would  appear  to  be  a 
uniform  density,  and  in  effect,  would  be  the  result  of  a  special 
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transformation  of  the  independent  variable.  The  logarithmic  interval 
histogram  is  one  such  transformation,  and  as  shown  in  Figure  3.2.1, 
does  produce  a  more  uniform  distribution  of  values  in  the  histogram 
intervals  Of  oourse  the  original  density  shape  is  lost  in  the  trans¬ 
formation,  but  as  we  uhall  see  later,  it  may  actually  be  easier  to  test 
"goodness  of  fit"  for  analytic  functions  on  the  transformed  variable 
than  on  the  original  variable.* 

The  data  in  any  realistic  experiment  is  of  finite  resolution, 
and  therefore  our  random  process  would  be  a  discrete  rather  than  a 
continuous  process.  This  discrete  nature  is  even  further  emphasized 
when  we  consider  the  data  in  histogram  form.  However-,  there  is  a  con¬ 
siderable  analytic  value  in  considering  the  continuous  case  since  it 
represents  the  envelope  of  the  discrete  function,  and  is  in  sane  cases 
more  easily  handled  in  the  mathematical  formulation.  We  will  actually 
consider  both  c^ses,  since  each  provides  a  certain  insight  into  the 
problem. 

3.2.2  Consideration  of  the  Log-Histogram  as  a  Transformation 

Variable 

A  oonparison  of  the  conventional  and  logarithmic  histograms 
in  Figure  3.2.1  lead  us  to  the  observation  that,  what,  ir  feet,  lad  hap¬ 
pened  was  a  transformation  of  the  independent  variable.  11.3  original 
exponential  density  function: 

* 

A  classical  goodness  of  fit  test  does  not  apply  outside  of  the  domain 
for  which  the  test  was  made.  However,  the  transformed  equivalent  of 
exponential-like  densities  have  more  significant  differences  in  ap¬ 
pearance  in  the  log-histogram  domain  than  in  the  conventional  scale 
domain. 
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(a)  THE  ORIGINAL  DENSITY  FUNCTION 


(b)  A  CONVENTIONAL  HISTOGRAM 


(c)  A  LOGARITHMIC  HISTOGRAM 

Figure  3.2.1.  A  Comparison  of  Conventional 
and  Logarithmic  Histograms 
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e(x;a)  =  a  exp  (-ax) 


0  <  x 


(3.1) 


was  transformed  into  a  new  function,  e(£;a)  by  the  transformation  of 
variables : 

£  =  log2*  (3.2) 

The  new  density  function  can  be  obtained  by: 

e(x;a)dx  =  e(£;a)d£  V  x,£  (3.3) 

which  yields: 

e (£;a)  =  e[x  =  exp  (£  In  2)]  •  || 
or: 

e(£;a)  =  a  exp  (-a  exp(£  In  2))  •  (In  2  exp(£  In  2) 

If  we  make  use  of  the  identity: 

a  =  exp(ln  a)  (3.4) 

we  then  obtain: 

e(£; a)  =  In  2  exp[(£  In  2  +  In  a)  -  exp(£  In  2  +  In  a)] 

(3.5a) 

We  note  in  passing  that  this  density  is  the  "doubly  exponential"  or 
Fisher-Tippett  type  1  density  (SA61),  which  is  also  known  as  the  dis¬ 
tribution  of  extreme  values,  A  form  which  more  closely  represents  the 
physical  probe lm  being  studied  can  be  obtained  by  converting  to  base 
two  exponentials. 

e(£;a)  =  In  2  exp[(£  +  £j-|)ln  2  -  exp((£  +  — -|)ln  2)  ] 
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or 


e(£?a)  =  In  2  exp2[(£  +  log2a)  -  jj^-exp2(£  +  l0(?2a^  (3*5fc>) 

This  equation  can  be  written  as  a  function  of  a  standardized 
variable  ip, 

ip  =  5  +  log2a  (3.6) 

resulting  in: 

eft»)  =  In  2  &xp2  exp2  (ip) )  (3.7) 

The  definition  of  ip  shews  that  it  is  merely  the  variable  £  plus  the 
constant  log2a,  i.e. ,  assuming  a  fixed  value  of  the  parameter  a. 
Therefore,  the  function  e( £;a)  is  of  the  same  shape  as  e(ip),  but  is 
shifted  along  the  £  axis  by  an  amount  equal  to  log2a.  This  observation 
has  seme  interesting  and  important  consequences . 

1.  The  transformed  exponential  will  have  exactly  the  same 
shape  regardless  of  its  mean  value. 

2.  The  effect  of  the  mean  is  merely  to  shift  the  function 
along  the  £  axis  by  log2a. 

3.  Fran  Theorem  3.2.1  we  shall  see  that  the  mean  value,  1/a, 
will  coincide  with  the  mode  of  the  transformed  density. 

4.  The  standardized  density  e(^)  will  have  its  modal  value 
at  ip  =  0. 

These  observations  are  shown  pictorially  in  Figure  3.2.2  which  shows 
the  original  exponential  density  and  both  the  transformed  density  and 
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(a)  THE  EXPONENTIAL  P.D.F. 


(c)  THE  NORMALIZED  TRANSFORMATION 


Figure  3.2.2.  Examples  of  the  Exponential  Density 
and  the  Transformed  Equivalents 
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the  standardized  version  of  the  transformed  function. 

Theorem  3.2.1 

The  modal  value  of  the  function  e(£;  a) ,  which  is  the  log- 
histogram  transform  of  the  exponential  density  function, 

where  is  related  to  the 
mean  value  of  the  original  density  function  by: 

^  =  'l092a 


e(x;a),  occurs  at  a  value, 


Proof: 

This  relationship  can  be  shown  by  setting  the  derivative  of  e(£;a)  to 
zero, 


^  e(C;a) 


_d 

d? 


a  In  2  exp[(£  In  2)  -  a  exp(£  In  2)] 


=  0 


such  that: 

a  In  2  {explain  2  -  a  exp^ln  2)]} 


•  [In  2  -  a  In  2  exp(£mln  2) ]  =  0 


Solving  for  we  have: 


a  exp  (5  In  2)  =  1 
m 

So  that: 

Sn  =  -l0^2a 

which  completes  the  proof  of  the  theorem. 
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Other  analytic  density  functions  could  be  transformed  in  a 
manner  similar  to  that  for  the  exponential.  However,  we  prefer  to  con¬ 
sider  only  the  hyperexponential  and  Erlang (K)  families  at  this  time, 
since  they  allow  a  mathematical  simplification  in  some  of  the  subse¬ 
quent  analysis.  These  two  families  of  functions  are  also  called  the 
Erlang  parallel  and  series  functions  respectively,  i.e,  they  are 
equivalent  to  a  parallel  or  cascade  group  of  exponential  functions.  In 
the  case  of  interest,  the  packet  lengths  may  be  modeled  by  such  density 
functions,  and  therefore,  so  would  the  resulting  service  time  density 
for  a  serial  transmission  facility. 

a.  The  Logarithmic  Transformation  for  Hyperexponential  Densities 

If  we  consider  the  two  component  hyper  exponential  density: 
h(x;alfa2)  -  Wj  [a^exp (-a.jX)  ]  +  w2 ^expf-a^)  ] ,  h  <_  x 

(3.9) 


where: 


w^  +  w2  =  1  and  w^ 

and  the  logarithmic  transformation: 

S  =  log2x 

we  obtain  the  transformed  density: 

n(C)  =  h(x  =  exp(£  In  2)) 


,w2  >  0 


dx 

'  ar 


t 
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so  that: 


n(S)  =  {w^a^expt-a^exp^  In  2)]  +  w2a2exp[-a9exp(C  In  2)]} 

•  {In  2  exp(£  In  2)} 
or: 

n(£)  =  w^{a^ln  2  exp  [£  In  2  -  a^exp(£;  In  2)]} 

+  w2(a2ln  2  exp[£  In  2  -  a2exp(£  In  2)]}  (3.10) 

\<!e  can  put  these  expressions  in  the  doubly  exponential  form  by  a 
manipulation  similar  to  that  of  equation  (3.5a)  and  obtain: 

n(£)  =  w^{ln  2  exp[  (In  a^  +  £  In  2)  -  exp  (In  a^  +  £  In  2)]} 

+  w2(ln  2  exp[  (In  a2  +  £  In  2)  -  exp(ln  a2  +  F,  In  2)  ] } 

(3.11) 

We  can  recognize  this  form  as  the  weighted  superposition  of  the  two 
exponential  components, 

n(£)  =w1e(C;a1)  +  w2e(£;a2)  (3.12) 

This  form  can  be  generalized  to  the  n  component  case,  namely: 
n 

n(S)  =  wi^(C;ai)  (3.13) 


n 

£  wi  =  1  31x1  W1  ,W2  "  '  "Wn  -  0 

i=l 
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The  weighted  superposition  can  be  visualized  as  the  density  resulting 
from  the  sum  of  n  of  the  e(ip)  functions,  each  of  which  has  been 
scaled  in  arrplitude  by  its  w^  weighting  and  shifted  by  the  l°g2ai 
value.  Seme  examples  of  such  hyperexponential  densities  and  their 
transformed  functions  are  shewn  in  Figure  3.2.3. 

b.  The  Logarithmic  Transformation  for  the  Erlang  (K)  Family 

A  second  family  of  "generalized"  density  functions  is  the 
set  of  Erlang  (K)  functions  which  can  be  used  to  model  densities  which 
have  a  variance  to  mean-square  ratio  of  less  than  unity,  i.e.,  whose 
standard  deviation  is  less  than  the  mean. 

The  general  form  for  this  family  of  density  function  is: 

g(x;a,k)  =  ■.  ,  ka (kax)k_1  exp(-kax),  0  <  x  (3.14) 

•  i  <  k 

where,  in  all  cases,  the  mean  is  1/a. 

For  k  =  1,  this  equation  reverts  back  to  the  exponential  For  k  =  2, 
it  is: 

g(x;a,2)  =  4a2x  exp(-2ax)  0  <_  x  (3.15) 

The  logarithmic  transformation  of  variables  of  Equation  (3.2),  with 
k  =  2,  produces: 

y(?;a,2)  =  4a2ln  2  exp[2£  In  2]  •  exp [-2a  exp (£  In  2)] 

(3.16) 

Sore  additional  algebraic  manipulation  and  the  use  of  the  identity  of 
Equation  (3.4)  result  in: 
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P  =  1/5 
7=1/4 


(a)  THE  ORIGINAL  TWO  COMPONENT  HYPEREXPONENTIAL  DENSITY 


(b)  THE  TRANSFORMED  DENSITY  CO  Y  ONENTS 


Figure  3.2.3.  The  Two  Component  Hyperexponential  Density 
and  the  Equivalent  Transformed  Components 
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y(5;a,2)  -  4  In  2  exp[2(ln  a  +  4  In  2)  -  2  exp(ln  a  +  4  In  2)  ] 
which  can  also  be  written  as : 

y(£;a,2)  =  4  In  2  exp  [2  In  2(4  +  log2a)  -  2  exp  (In  2(4  +  iog2a) )  ] 

If  we  make  the  substitution  of  variables  of  Equation  (3.6),  and  con¬ 
vert  to  base  two  exponentials,  we  obtain: 

y(*;2)  =  4  In  2  |exp2ty  -  jJl-  exp2(*)]|2  (3.17) 

for: 

exp2(y)  =  2y  =  exp(y  In  2) 

and: 

=  4  +  log2a 

We  can  catpare  this  result  with  the  exponential  case  of  Equation  (3.7), 
which  is  also  a  member  of  the  Erlang  family, 

Y(^;l)  =  e(4»)  =  In  2  exp2[<|>  -  ~j  exp2  ty>)  J 

Two  observations  can  be  made  from  the  comparison.  First,  a  generaliza¬ 
tion  seems  to  be  readily  available,  and  secondly,  the  functions  both 
have  their  modal  points  at  the  transformed  values  of  the  mean.  The 
latter  observation  is  apparent  because  only  the  exponent  portion  of  the 
expression  can  be  equal  to  zero,  and  this  portion  is  the  same  in  both 
equations . 

The  general  case  for  the  transformed  function  can  be  shown 

to  be: 
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y(£;a,k)  =  .'‘yyy  {exp [(In  a  +  £  In  2)  -  exp(ln  a  +  £  In  2) ] f 
or  in  normalized  form: 


Y(<Jpk)  =  exp2[i^ 


InT  exp2  GJ>)  ] 


(3.18) 


We  can  find  the  modal  point  by  taking  the  derivative: 


^y(*;k)  =k 


’(k)kln  2 

_d 

_(k  -  1) ! 

...  | 

d\|j 

exp2^  "  exP2(^)' 


The  only  part  of  the  resulting  derivative  which  can  be  equal  to  zero  is 

a|  t*  -  Err exp2(WI  =  0 

which  results  in: 

V  “  +  la>2a  =  n 


or: 

-  -log2a  (3.19) 

This  result  is  a  particularly  interesting  generalization  for  the  entire 
Erlang (K)  family.  The  node  of  the  transformed  density  will  occur  at 
the  transformed  equivalent  of  the  mean,  1/a,  and  as  seen  for  the  ex¬ 
ponential  case,  changes  in  the  mean  will  merely  shift  the  function 
along  the  K  axis. 

c.  The  Logarithmic  Transformation  of  the  Geometric  p.m.f. 

The  geometric  probability  mass  function  is  of  special 
interest  since  it  is  the  discrete  analog  of  the  exponential  density, 
and  thereby  has  the  well  known  memory  less  property  which  is  so  useful 


78 


and  simplifying  in  analysis.  Fortunately,  these  two  functions  are 
reasonable  models  for  many  real-v/orld  processes,  e.g.,  processes  in 
which  we  are  interested  in  the  time  (or  number  of  berroulli  trials)  be¬ 
fore  an  event  will  occur.  In  our  particular  application,  that  "event" 
is  the  occurrence  of  the  "end  of  packet"  terminator. 

The  geometric  p.m.f.  can  be  considered  as  the  probabilities 
associated  with  a  series  of  bemoulli  trials.  The  trials  are  repeated 
until  a  success  is  achieved,  with  a  probability  of  success  equal  to  q 
on  each  trial.  The  probability  of  a  success,  preceded  by  n  failures 
is  the  classical  geometric  p.m.f.; 

p(n;q)  =  q(l  -  q)n  n  =  0,1,2,...  (3.20) 

In  our  application,  the  "success"  is  the  occurrence  of  the  special 
terminator  symbols,  which  will,  in  all  cases,  be  preceded  by  at  least 
one  word  of  message  content.  Therefore,  we  require  that: 

p(0;q)  =  0 

so  that  we  must  renormalize  the  p.m.f.  by: 

P(n;q)  =  r^ToTqT  [q(1  "  q)n]  n  =  1,2,3,... 

Since  we  know  that  p(0;q)  is  equal  to  q  we  have: 

p(n;q)  =  q(l  -  q)n_1  n  =  1,2,3,...  (3.21) 

This  new  function  is  merely  the  original  geometric  p.m.f.  shifted  to 
the  right  one  integer  value  in  n,  and  serves  as  an  example  of  the 
memoryless  property.  That  is,  the  conditional  p.m.f.,  ,i.ven  that  the 
length  is  greater  than  zero,  is  still  a  geometric  p.m.f.,  as  would  be 
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the  case  for  the  conditional  p.ra.f.  given  that  the  length  is  greater 
than  any  arbitrary  value,  n. 

The  transformation  which  we  wish  to  consider  causes  the  data 
to  be  grouped  into  a  logarithmic  histogram  according  to  the  most  sig¬ 
nificant  bit  of  the  binary  data  representation.  Therefore,  one  interval 
will  contain  the  occurrences  of  n  =  1,  the  next  will  contain  the 
occurrence  of  n  =  2  or  3,  the  next  the  occurrences  of  n  =  4,5,6,  or  7, 
etc.  The  probabilities  of  each  of  these  intervals  is  therefore: 

tt  (0  ;q)  =  p(l)  =  q 

it (l;q)  =  p(2)  +  p (3)  =  q(l  -  q)  +  q(i  -  q)  2 

tt  (2  ;q)  =  p(4)  +  p(5)  +  p  (6)  +  p(7)  =  q(l  -  q) 3  +  ...  +  q(l-q)6 

Each  of  these  expressions  can  be  considered  to  be  a  truncated  geometric 
series,  resulting  in  the  set  of  equations: 

Tr(0;q)  =  q 

tt(1  -  q)  [1  -  (1  -  q)2] 

TT (2  ;q)  =  (1  -  q)  3  [1  -  (1-q)4] 
or  in  general : 


2V-1  2V 

7T(v;q)  =  (1  -  qr  [l  -  (1  -  qP  'J 


(3.22) 


For  ease  of  computation,  a  recursive  form  of  the  equations  is  more 
convenient,  namely: 


-.V-l 


Tr(v;q)  =  tt(v  -  l,q)  •  (1-q)' 


iV-1 


1  +  (1  -  q)‘ 


(3.23) 


The  discrete  function  7r(v;q)  is  analogous  to  the  continuous  function, 
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e(£;a),  in  the  previous  analysis.  However,  in  the  7r(v;q)  case,  the 
mode  is  not  so  precisely  1 oca table,  and  hence  is  not  such  a  good  in¬ 
dicator  of  the  average  value.  Fortunately,  there  is  another  indicator 
for  the  discrete  case,  as  follows: 

ir(0;q)  *  q, 

and  since  we  know  that, 

E [n]  -  n  =  1/q, 
we  therefore  have: 

n  =  l/tr(0;q) .  (3.24) 

3.2.3  Approximating  Density  Functions 

The  preceding  section  considered  lo  [-histogram  representations 
of  density  functions,  and  discussed  a  number  of  their  properties  and 
characteristic  shapes .  These  histogram  shape  differences  are  both  an 
asset  and  a  liability,  as  shewn  in  Figure  3.2.4.  In  this  exanple  the 
hyperexponential  and  truncated  1/x  densities  are  shewn  to  have 
significantly  different  log-histogram  shapes,  but  differ  only  slightly 
in  their  uniform  interval  histograms,*  due  to  the  poor  resolution  near 
the  origin.  However,  the  uniform  histogram  has  the  intuitive  advantage 
of  approximately  resembling  the  general  shape  of  the  density  being 
approximated,  while  the  log-histograms  have  shapes  which  are  initially 
semewhat  peculiar.  However,  with  seme  experience  in  using  then,  these 
log-histogram  shapes  become  readily  recognizable,  and  are  very  useful 

^Histograms  with  uniform,  i.e.,  equally  spaced,  intervals  will  be 
referred  to  as  "uniform  histograms"  in  the  sequel. 
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for  distinguishing  between  different  density  functions  such  as  the  two 
shown  in  Figure  3.2.4.  Part  (c)  of  that  figure  shows  two  scales  on  the 
log-histogram,  and  leads  to  an  alternative  representation  which  has  the 
best  features  of  both  types  of  histograms  as  shewn  in  Figures  3.2.5  and 
3.2.6,  where  these  same  two  log-histograms  are  shewn  with  a  rescaled 
probability  axis. 

The  rescaling  is  based  on  the  probability  area  preserving 
property  of  the  transformation  of  variables, 

f  (x)dx  =  4>(£)d£  (3.25) 

which  makes  quantiles  of  the  density  invariant  to  the  transformation, 
and  therefore  allows  one  to  transform  a  density  function  in  a  piece- 
wise  fashion.*  That  is,  the  component  of  the  4>(S)  density  function 
between  ^  and  can  be  transformed  into  the  equivalent  conponent 
of  f(x)  between  x^  and  x^+1,  where: 


xi  * 


(3.25) 


is  the  appropriate  transformation  of  variables. 

Because  such  a  transformation  must  preserve  the  probability 
area,  we  have  the  discrete  equivalent  of  Equation  (3.25) , 


Wi  -  =  pi(xi+i  •  V 


(3.27) 


The  k  quantile  of  an  arbitrary  density  function,  p(y) ,  is  the  value, 
for  which: 


— 00 


p(y)dy  =  k 


0  <  k  <  1 
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p(x)  ■  1/x  In  150 


Figure  3.2.5.  The  1/x  Density  Function  and  its  Approximation 
from  the  Log-Histogram  Data 


Figure  3.2.6.  The  Hyperexponential  Density  (p  *  1/15  and  7*  1/3) 
and  its  Approximation  from  the  Loo- Histogram  Data 


where  ard  and  the  density  values  in  the  intervals  of  the 
and  x  domains  respectively.  For  the  general  log-histogram  trans¬ 


formation  of  variables, 
xi  =  (b)  i 


(3.28) 


and: 


=  i,  i  =  0,1,2, ... ,N-1, 


which  results  in  the  simplification: 


-  h  *  1 


(3.29) 


and  the  density  rescaling  becomes: 

p.  =  ^/[(b  -  1)  (b)1]  (3.30) 

Figures  3.2.5  and  3.2.6  show  that  such  area  transformations 
cb  indeed  produce  estimates  of  the  density  function  which  closely  model 
the  true  density  for  the  hyper  exponential  and  truncated  1/x  densities, 
and  in  Figure  3.2.7  a  similar  fit  is  noted  for  the  Erlang (2)  density. 
Figure  3.2.8  shows  the  log-histogram  form  of  the  Erlang (2)  density, 
and  one  can  see  that  the  function  is  somewhat  more  peaked  than  that  for 
the  exponential,  and  that  the  mode  corresponds  to  the  transformed  value 
of  the  mean  as  shown  in  Section  3.2.2  (b) . 

These  examples  shew  that  for  the  exponential- like  densities 
of  interest,  i.e. ,  densities  with  most  of  the  probability  area  near  the 
origin  and  with  a  long  "tail"  extending  over  a  wide  range,  that  log- 
histograms  provide  an  effective  measurement  technique,  and  require  a 
relatively  snail  number  of  histogram  segments.  Although  no  claim  is 
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Figure  3.2.7.  A  Comparison  of  the  Erlang (2)  Density,  (x  “  20) 
and  its  Log-Histogram  Approximation 


Figure  3.2.8.  A  Log-Histogram  for  the  Erlang(2)  Density  Function 


86 


male  that  log-histograms  are  appropriate  for  arbitrary  density 
functions,  these  exanples  do  cover  a  reasonable  range  of  skewed  den¬ 
sities,  and  in  the  Erlang  (2)  case,  differs  considerably  frcm  the 
"equal  interval  area'1  criterion  expressed  in  Section  3.2.1.  For  den¬ 
sities  with  sane  other  general  shape,  another  transformation  of  vari¬ 
able  may  give  similar  improvements,  and  much  of  the  resulting  analysis 
would  be  similar  to  that  presented  here  and  in  the  moment  estimation 

aspect  considered  in  Section  3.3. 

A  variation  of  the  ccmponent-by-canponent  transformation  can 
be  obtained  if  we  consider  each  histogram  interval  in  the  £  domain  to 
be  a  uniform  density  segment,  and  transform  these  uniform  segments 
into  their  equivalent  f  (x)  components ,  as  indicated  in  the  following 
theorem. 

Theorem  3.2.2 

The  uniform  density  segment, 

*  i<J<i  +  l 

is  the  transform  equivalent  of  the  segment: 

f^x)  =  In  b  <3*31 

for  the  transformation  of  variables, 
x  =  (b)C 

Proof: 

The  1th  segnent  of  a  log-histogram  has  a  functional  form  of : 

♦  •(£)  =  tt.  ,  i<5<i  +  l 
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such  that  for  the  transformation  of  variables, 
x  -  (b)€ 

and: 

f  (x)dx  =  ♦(Od? 

we  have  the  approximating  function: 

£>>  =hK)  i 

or: 

f^  (x)  =  -n\  /x  in  b  (b)^  <_  x  <  (b)^+^  (3.32) 

Thus,  the  density  f  (x)  is  approximated  by  a  set  of  truncated  1/x 
density  segments  which  are  weighted  by  the  ir  probabilities.  This 
result  proves  the  thec^m,  but  unfortunately  proves  an  asthetically 
poor  approximation  to  the  f  (x)  function  due  to  the  sharp  cusps  at  the 
beginning  of  each  density  segment.  The  technique  would  be  increasingly 
worse  as  the  density’  deported  further  frem  the  1/x  shape,  and  therefore 
was  not  seriously  considered  as  a  density  modeling  technique.  However , 
a  variation  of  the  irethod  is  pursued  in  Section  3. 3. 2. 2  for  estimating 
moments. 

3.3  intimation  of  Moments 

Theoretically  all  of  the  information  about  a  simple  randan 
variable  is  given  by  either  its  density  function,  the  cumulative  dis¬ 
tribution  function,  the  characteristic  function,  or  the  set  of  all 
rrerrents.  The  latter  is  infinite  in  number,  but  fortunately  the  higher 
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order  moments  are  of  rapidly  decreasing  interest.  Indeed,  for  much 
analysis  work,  only  the  first  one  or  two  moments  are  of  concern.  Fo1' 
example,  in  same  processes  the  expected  performance  of  the  system  can 
be  obtained  from  given  average  values  of  the  variables,  while  in  others , 
such  as  the  queueing  system  considered  in  the  Pollaczek-Khinchine 
equation  (SA61) , 


E[n] 


P2  ♦  X2o  2 

Xx  +  2(1  =-5V“ 


(3.33) 


both  the  first  and  second  moments  of  the  dependent  variable  are  neces¬ 
sary  to  predict  the  average  system  behavior.  In  both  cases,  this 
analytical  tract  ability  is  the  basic  reason  that  moments  play  such  a 
prominent  role  in  analysis  work,  rather  than  other  sets  of  parameters 
such  as  quantiles.  In  the  latter  case,  there  is  no  analogous  basis  for 
combining  random  variables  in  even  such  a  simple  case  as: 

E[x  +  y]  =  E[x]  +  E[y] 

due  to  the  mathematically  awkward  definition  of  quantiles;  where  as 
previously  defined,  the  k^  quantile  is  at  the  value  for  which: 


/ 


p(y)dy  =  k 


0  <  k  <*  1 


In  contrast,  the  n^  moment  is  defined  as: 


(3.34) 


OD 

E[xn]  =  J  xn  p(x)dx 

— oo 


(3.35) 
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for  a  continuous  density  function,  p(x),  or  in  general: 


E[xn]  =  / xn  d  P(x)  (3.36) 

•OO 

This  latter  form  is  the  Stieltjes  integral  form  which  also  includes 
discontinuous  and  discrete  functions.  A  similar  definition  applies  to 
nments  about  a  point,  a, 

00 

E[(x-a)n]  =  f  (x  -  a)n  d  P(x)  (3.37) 

— 00 

which  are  the  central  moments  for  the  usual  case  of  a  =  E  [x] . 

3.3.1  A  Comparison  of  Moment  Estimation  Techniques 

The  most  straightforward  method  of  estimating  the  moments  of 

a  population  is  the  classical  approach, 

_  r  n.  1  v*  n 
E [x  ]  =  six 

lN  N 

This  method  is  quite  satisfactory  for  estimates  of  the  mean  because 
only  the  summat i  on  is  necessary  in  that  case .  However ,  the  requirement 
to  raise  each  measured  data  value  to  a  Dcwer  for  the  higher  moments  nay 
be  infeasible  in  some  applications.  In  the  network  measurement  example, 
the  operation  of  the  IMP  would  have  been  degraded  by  the  calculation 
necessary  for  even  the  second  moment  due  to  the  lack  of  multiplication 
hardware,  and  due  to  the  possibility  of  register  overflew  unless  doublr- 
precision  arithmetic  were  used.  These  lengthy  processing  requirements 
were  not  felt  to  be  consistent  with  the  need  for  real  time  message 
handling  capabilities,  so  other  techniques  were  investigated. 
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A  viable  alternative  method  to  estimate  the  moments  is  the 
use  of  grouped  data  accumulation  procedures,  the  best  known  of  which  is 
the  simple  uniform  histogram.  Other  variations  are  the  log-histooram 
and  the  quantile-gram  which  were  intended  to  have  more  optimal  interval 
sizes  based  on  the  equal  probability  area  criterion.  However,  a  better 
criteria  for  the  moment  estimates  would  be  to  have  intervals  with  ap- 

9 

proximately  equal  x  •  p (x)  or  x  •  p (x)  values ,  which  would  typically 

require  narrower  intervals  at  higner  values  of  x;  which  are  just  the 

opposite  of  the  interval  variation  which  were  desired  for  density 

estimates.  The  desire  for  optimal  interval  sizes  based  on  not  only 

2 

equal  probability  areas,  but  also  equal  x  p(x)  and  x  p(x)  ocmponents 
would  require  three  separate  sets  of  grouped  data.  However,  we  will 
find  that  for  our  purposes,  histograms  provide  an  appropriate  oonpro- 
mise  between  these  needs.  Ihe  contribution  of  each  such  histogram 
component  to  the  probability  area  and  to  the  first  and  second  moments 
of  an  exponential  density  is  shewn  in  Tables  3.3.1(a)  and  (b)  for  both 
uniform  and  log-histograms.  While  none  of  the  estimation  criteria  are 
optimal,  the  histograms  do  provide  a  reasonable  balance  of  the  three 
estimation  needs.* 

3.3.2  Methods  of  Estimating  Moments  from  Log-Histogram  Data 

Several  different  methods  are  possible  for  estimating  the 
moments  of  the  actual  density  function,  f (x) ,  from  the  log-histogram 
data.  For  exanple,  if  we  consider  the  transformation  of  variable 

The  area  and  moment  totals  for  the  two  tables  are  exactly  the  same 
since  the  component  values  are  calculated  trom  the  moment  integrals. 
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TABLE  3.3.1 


COMPONENTS  OF  THE  DENSITY  AND  FIRST  AND  SECOND  MOMENTS 
FOR  LOG  AND  UNIFORM  HISTOGRAM  INTERVALS  (EXPONENTIAL  DENSITY) 

a.  Log-Histogram  Intervals 


t 

1 

Interval 

I 

1— 

Pr [xel] 

fx  p(x)dx 

I 

f2 

Jx  p(x)dx 

I 

0-1 

.061 

.05 

.5 

1-2 

.056 

00 

o 

• 

.5 

2-4 

.104 

.29 

1.5 

4-8 

.173 

1.05 

5.1 

8-16 

.238 

2.76 

33.2 

16-32 

.233 

5.31 

125.7 

32-64 

.117 

5.01 

225.5 

64-128 

.018 

1.42 

114.0 

!  Totals 

1.000 

15.97 

506.0 

b.  Uniform  Histogram  Intervals 


i — 

J 

Interval 

I 

Pr  [xel] 

Jx  p(x)dx 

/x2p  (x)dx 

I 

' 

0-16 

.632 

4.23 

40.8 

16-32 

.233 

5.31 

125.7 

32-48 

.085 

3.26 

127.5 

48-64 

.032 

1.75 

198.0 

64-80 

.011 

.77 

55.0 

80-96 

.004 

.38 

33.0 

96-112 

.002 

.20 

17.0 

112-128 

.001 

.07 

9.0 

Totals 

1.000 

15.97 

506.0 
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approach,  we  can  compute  the  moments  of  the  (£)  function  and  relate 
the  two  sets  of  moments.  Alternatively,  we  can  transform  each  element 
of  the  log-histoqram  into  an  equivalent  f(x)  elemental  ccnponent,  or 
finally,  we  can  make  a  simple  area  transformation.  In  the  latter  case, 
a  considerable  refinement  of  the  estimates  can  be  made  by  introducing  a 
first-order  approximation  to  the  slope  of  the  density  function  over 
each  interval.  Each  of  these  techniques  shall  be  considered  in  detail 
in  the  following  sections. 

3. 3.2.1  Moment  Transformations 

If  data  sanples  are  accumulated  in  a  log-histogram,  we  can 
consider  the  resulting  information  to  be  in  the  transformed  or  £ 
domain,  and  can  therefore  estimate  the  moments  of  $(£).  Of  course,  we 
are  actually  concerned  with  the  moments  of  f  (x) ,  and  therefore  need 
seme  relation  on  transfer, nation  between  the  two  sets  of  moments .  Such 
a  relation  is  shown  in  the  corollaries  to  the  following  theorem. 

Theorem  3.3.1 

The  moments  of  f  (x)  can  be  related  to  those  or  <MS)  by: 

Ef  [xn]  *  ^(jw  *  n  In  b)  *  E^[en^lnb]  (3.39) 

where  $  (jw)  i£  the  characteristic  function  of  the 

V 

density,  #(£)  # 

oo 

(jw)  =  y*exp[jw£]<M£)d£  (3.40) 

— OO 

and  the  transformation  of  variables  has  been  generalized  to: 
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Proof: 


C  = 


The  n  mcment  of  f(x)  is: 


0  <  x 


xn  *  J*° xnf(x)dx 
0 

which  for  the  logarithmic  transformation  of  variables, 


K  =  log^x  x  =  exp  (?  In  b) 


becanes: 


_  £=+» 

xn=  /  (e 


{exp(C  In  b)  }n[f  (x  =  exp(£  In  b))^r]d£ 


We  can  recognize  the  term  in  the  brackets  as  <j>  (O  ,  so  we  have: 


exp( n  £  In  b)<f>(S)d£ 


The  integral  expression  is  identical  to  the  definition  of  the  charac¬ 
teristic  function  if  we  replace  jw  by  n  In  2.  Therefore,  we  have: 


xn  =  (jw  =  n  In  2) 

which  ocmplebes  the  proof  of  the  theorem. 

The  characteristic  function  contains  all  the  mcment  informa¬ 
tion;  indeed  a  similar  development  using  a  real  variable  exponent  is 
called  the  mcment  generating  function.  We  can  modify  the  result  of  the 
theorem  to  make  the  moments  explicit  by  expanding  the  exponential  term 
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as  shown  in  the  following  corollary. 

Corollary  1 

The  relationship  between  the  moments  of  f  (x)  and  those  of 
$(£)  is: 

Ef  txn]  *  f)  (n  &  b)1  (3.«) 

i=0 

Proof: 

The  exponential  term  can  be  expanded  in  a  power  series , 

exp[n  £  In  b]  =  1  +  n  £  In  b  +  ^n  —  +  •  •  •  (3.43) 

which  when  substituted  into  Equation  (3.39),  results  in: 


?-/ 


[1  +  (n  £  In  2) 


+  (n  ^  21-  +  ...]  4>(OdC 


from  which  we  obtain: 


/oo 

$(C) d£  +  n  In  2  J  5  <M5)d5  + 


or 


xn  =  l  +  nln2£  + 


(n  In  2 )l_  ^2  + 


21 


(n  In  2; 3  ^3  + 


31 


(3.44) 


which  catpletes  the  proof  of  the  corollary . 

Equation  (3.44)  literally  says  that  we  need  to  know  all  of 
the  moments  of  $  (£)  to  calculate  any  one  (or  all)  of  the  moments  of 
f  (x) .  In  practice,  we  would  expect  that  the  first  few  moments  of  4>(5) 
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would  be  adequate  to  estimate  the  mean  and  variance  of  f  (x) .  Unfor¬ 
tunately,  the  series  does  not  converge  very  rapidly,  and  is  therefore 
of  little  general  value.  An  example  usage  of  the  equation  is  shorn  in 
Appendix  C,  in  which  $(£)  is  the  uniform  density  function  correspond¬ 
ing  to  f  (x)  =  1/x  In  M.  The  example  demonstrates  the  poor  convergence 
since  even  after  considering  the  first  five  moments  of  <J>  (O  the 
estimate  of  x  is  still  in  error  by  26%. 

A  variation  of  the  moment  expansion  which  results  in  improved 
convergence  properties  involves  the  central  moments  of  <P  (O  >  and  is 
developed  in  the  following  corollary. 

Corollary  2 

Itie  moments  of  f  (x)  can  be  found  in  terms  of  the  central 

moments  of  <J>(5)  by: 

Ef[xn]  =  exp[n  X  In  bill  +  ]|P  (n  E  [g  -  X)n)  |  (3.45) 

i=2 


Proof: 


If  we  consider  the  transformation  of  variables  to  be  the  function. 


x  *  g(C)  =  exp[£  In  b]  (3.46) 

and  expand  the  function  in  a  Taylor  series  about  the  mean,* 

2 

g(0  «  g(0  +  (€  -  E)g'  (X)  +  g" (0  +  ...  (3.47) 


This  approach  is  similar  to  that  considered  by  Papoulis  (PA658) , 
pp.  151-152,  in  which  he  approximates  the  expected  value  of  g(z)  in 
terms  of  the  first  few  moments  of  the  density  function  for  z. 
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so  that  when  we  take  expected  values,  we  obtain: 


x  =  g(0  +  -  T\  g'(Q  +  -  C)2]  g"(U  +  ... 

He  then  have  an  expression  for  x  in  terms  of  the  central  moments  of 
<p(0 ,  which  when  we  make  the  substitution, 

g°°  (O  =  (In  2)kexp(C  In  2) 

we  obtain: 

2  3 

x  =  exp {Z  In  2){1  +  P2  ^2^  ~  +  ^3  ^3^  ’  +  •••)  (3.48) 

where  is  the  k  central  mcment  of  <j>(C)»  e.g., 


0 


o 


2 

K 


For  the  exponential  transformation  of  interest,  the  method  can  be  ex¬ 
tended  to  higher  moments  of  f  (x)  by  considering: 

xn  =  [g(£)Jn  =  exp[n  £  In  2)  =  g(n  £)  (3.49) 

so  that  the  Taylor  series  expansion  about  the  mean  becomes, 

g(n  K)  =  g(n  D  +  n(£  -  O  g'lnO  +  ... 


We  need  to  consider  the  derivatives  a  bit  more  carefully  at  this  point; 


g00  (n  0 


d(n  £> 


^  exp(n  f.  In  2) 


=  l 
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or: 


g(k)  (n  O  =  (In  2)k  exp(n  Ft  In  2) 

Therefore,  the  expansion  becomes: 

g(n  O  =  exp(n  £  In  2)  +  n(£  -  O  In  2  exp(n  X  In  2)  +  ... 

g(n  0  «  exp(n  £  Ln  2){1  +  n  In  2(£  -  £)  +  (n  —  (£  -  £)2  +...} 

When  we  take  expected  values,  we  obtain: 

xn  =  exp(n  X  In  2){1  +  (-n-^ -2)  v2  +  — -jy-2-  ■  U3  +  ...}  (3.50) 

which  oorrpletes  the  proof  of  the  corollary. 

The  analysis  also  considered  other  forms  of  moment-like 
parameters  including  cunulants,  which  are  related  to  the  central 
moments  by: 


k2  =  U2  '  (ui)2  =  °2 

2 

k3  =  m3  *  3u2U1  +  2^) 
etc. , 

or  in  general  by  their  respective  generating  functions: 

K (t)  =  In  4>(t)  (3.51) 

The  cumulants  are  also  known  as  the  semi-invariants  since,  with  the 
exception  of  k^,  they  are  invariant  to  the  translation  of  the  density 
along  the  axis.  The  use  of  omulants  in  selected  test  case  examples 
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resulted  in  convergence  properties  similar  to  that  observed  for  the 
central  moments,  and  since  more  viable  moment  estimation  approaches 
were  found,  no  further  investigation  of  the  moment  trans  forma  Lion  i[>- 
p roach  was  pursued.  These  other  techniques  will  be  described  in  the 
following  sections. 

3. 3. 2. 2  Superposition  of  Transformed  Histogram  Segments 

We  can  find  an  alternative  method  of  estimating  the  moments 
by  an  extension  of  the  results  of  Theorem  3.2.2.  This  theorem  shewed 
that  a  uniformly  shaped  0  (O  corresponds  to  a  truncated  1/x  density 
function  for  f  (x) ,  and  that  translating  a  uniform  density  segment 
along  the  £  axis  was  equivalent  to  selecting  a  new  segment  of  the 
1/x  density,  and  weighting  it  by  the  probability  ir. 

If  we  consider  a  histogram  in  the  £  domain  to  be  a  set  of 
uniform  density  components ,  we  can  then  apply  the  theorem  result  to 
ccmpute  a  corresponding  set  of  properly  scaled  1/x  components.  The 
composite  function  in  the  x  domain  would  rot  be  aesthetically  pleas¬ 
ing  due  to  the  discontinuities  and  sharply  peaked  cusps  at  the  begin¬ 
ning  of  each  1/x  density  segment.  However,  tho'e  spikes  do  not 
necessarily  oontain  sufficient  area  to  affect  the  mcment  estimates 
which  are  the  real  objective  of  the  method.  We  can  find  tie  mcment 
estimates  by  the  following  theorem. 

Theorem  3.3.2 

The  moments  of  the  distribution  in  the  x  domain  are  related 
to  the  log-histograp  interval  probabilities,  rr  by  the 
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relationship: 


-dfe]  t 


(3.52) 


where  b  is  the  base  of  the  exponential  transformation,  such 
that: 


x  =  (b) 


(3.53a) 


=  Pr[i  <  i+l] 


Proof: 


We  know  from  Theorem  3.2.2  that  for  the  ocnponent: 


(3.53b) 


<t>i  (0  *  ^ 


i  <  C  <  i+l 


the  corresponding  segment  in  the  x  domain  is: 


f^fx)  *  iTj/x  In  b 


(b)1  <  x  <  (b)i+1 


(3.54) 


This  ocrponent  will  have  a  contribution  to  the  n  moment  of  f  (x)  of : 


Ef  [x^  |  the  i^  carpcnent] 


/.i+l 

>  A, 

x  In  1 


Ef[xn  |  the  1th  carponent]  =  n'  ^  [(b)1^  ( (b) n  -  l)j 


ino 


which  when  all  such  ocrponents  are  considered,  becanes: 


Ef[xn]  =  £ 


N 

L 

i=0 


IT. 


;nj-E[(b)ni((b)n 


or: 


“[w!r]  t 

i=0 


m 


which  is  equivalent  to  assuming  that  all  of  the  probability  mass,  tk  , 
is  located  at  5  =  0,1,2,...,^,...,  and  correcting  for  this  offset  by 
the  coefficient  outside  the  surma tion .  This  ocrpietes  the  proof  of 
Theorem  3.3.2,  and  we  shall  later  consider  seme  examples  of  this  method 
of  estimating  manents,  for  the  special  case  of  b  equal  to  two,  i.e., 
the  bast  two  histograms  which  are  most  convenient  with  binary  computers. 
The  above  theorem  considered  the  general  case  since  it  requires  little 
additional  effort,  and  because  it  will  be  of  value  in  subsequent 
analysis . 

Theorem  3.3.2  is  valid  for  V  <  K  which  corresponds  to 
1  £  x.  Since  the  log-histogram  method  is  also  viable  for  0  <  x  ,  we 
need  to  extend  the  moment  estimate  to  include  this  interval  to  be 
mathematically  oociplete.  (In  the  case  of  integer  values  of  x,  the 
only  value  to  be  added  is  zero,  which  has  no  contribution  to  the  mo¬ 
ments.  However,  for  seme  of  our  test  cases,  we  consider  a  continuum  of 
values  of  x.  and  such  values  might  also  be  found  in  seme  other  appli¬ 
cation  of  the  technique . ) 
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Corollary  1 


til 

Itie  interval  0  <  x  <  1  has  a  contribution  to  the  n 
moment  of  approximate]/: 


E[xn  |0  <  x  <  1]  S 


(b)n  -  1 
(b)wl  - 1 


TT_ 

n  In  b 


(3.56) 


where  tt_  is  the  probability  for  the  corresponding  interval 
in  the  £  domain. 

Proof: 

We  can  not  utilize  a  1/x  density  in  the  region  0  <  x  <  1  because 
the  resulting  area  does  not  remain  finite.  Instead,  we  will  assume  a 
uniform  density  in  the  x  domain  for  that  region,  and  knowing  that  the 
probability  components  in  the  £  domain  cure  (from  Equations  (3.53a) 
and  (3.53b)): 


ir_  +  tt_2  +  +  ... 

which  to  maintain  a  uniform  density  in  x,  must  be: 


ii 


+ 


•  •  • 


or 


b 

=  7r— i  E^~r 


(3.57) 


such  that: 
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Since  the  theorem  results  are  applicable  for  both  positive  and  negative 
values  of  the  index,  we  have: 


E[x 


n 


due  to  0  <  x  < 


« -  [-fe1]  t  *.i 

i=l 


(b) 


-m. 


(b)n  -  1 
n  In  b 


-ni 


which  simplifies  to: 


E[x  |  due  to  0  <  ::  <  1] 


(b)n  -  1  .  ^ 
,  .n+1  _  ,  n  In  b 


(3.58) 


arxi  therefore  proves  the  corollary. 

Corollary  2 

The  theorem  result  is  the  discrete  analog  of  that  of 
Theorem  3.3.1,  such  that: 


lim 

AC-0 


00 

^i (b)ru  =  J  <))(0enClTlbdC 
0 


(3.59) 


where  AC  is  the  histogram  interval. 

Proof: 

The  resolution  of  a  log-histogram  is  inversely  proportioned  to  the  base 
of  the  logarithms,  since  the  intervals  in  the  x  domain  are  bounded  by 
the  values  b,b2,b3,...  etc.  Therefore,  as  one  allows  the  value  of  b 
to  approach  unity,  the  histogram  intervals  become  arbitrarily  small, 
and  the  sun  approaches  an  integral  form,  with  the  probability  element 
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E 


Hie  limit  then 


ir  being  replaoed  by  the  differential  element  $ (£)d£. 
becomes: 


lim  Ef [xn] 
A£-*0  r 


lim 

b+1 


(b?n  -  1  1  ^ 
n  In  b  j  Z~» 
i=0 


iri  (b) 


ni 


und  since  one  can  show  by  L' Hospital's  rule  that: 


lim 

b*l 


(b)n  -  1 
n  In  b 


=  1 


we  are  left  with  the  limit: 


N 

lim  Ef[xn]  =  lim  V  n  (b)n^ 
A£->0  W  g  ^ 

N-H« 


(3.60) 


where  we  have  taken  advantage  of  the  result  of  Theorem  3.3.2  that 
allows  one  to  aonsider  all  of  the  probability  mass  to  be  located  at 
£  =  i.  We  can  then  write  the  exponential  term  as: 


(b) 


n£ 


en£lnb 


which  we  can  show  to  be  invariant  to  the  limit  process  since  for  any 
given  value  of  the  variable  x  the  value  of  £  is: 


£  =  log^x  =  In  x/ln  b 

such  that: 


£  In  b  =  In  x  (3.61) 

and  therefore,  for  a  given  value  of  x,  the  product  £  In  b  is  fixed. 
The  overall  limit  then  becomes  the  integral, 
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lim 

AC-K) 


oc 

Ef[xn]  =  f  *(C)enClllbd5 
0 


(3.62) 


wtuch  is  the  result  iron  Theorem  3.3.1.  The  analogy  is  further  indi¬ 
cated  by  noting  that  the  original  form  of  the  sun  can  be  considered  to 
be  a  probability  generating  function; 


E(xn]  =  [  n'LVjnt2  =  fan)  (3.63) 

where: 

N 

n(7)  =  ^irkzk  (3.64) 

k=0 


and  this  probability  generating  function  can  be  considered  to  be  the 
discrete  equivalent  of  the  characteristic  function  (or  its  more  simi¬ 
larly  named  variation,  the  moment  generating  function) . 

Corollary  3 

The  result  of  Theorem  3.3.2  produces  estimates  of  the  moments 
which  are  asymptotically  unbiased. 

Proof: 

The  moment  estimate  from  Theorem  3.3.2  was: 

n  N 

E(xn  |  log  histogram]  =  £  ^(b)^ 

i=0 

which  was  shown  in  the  limit  to  be  equivalent  to  the  continuous  result 
of  Theorem  3.3.1.  Therefore,  the  limiting  value  of  the  estimate  is 
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i 


unbiased  as  the  interval  size  decreases  toward  zero  since: 

E[xn  |  log-histogram]  -  E[xn] 

Corollary  4 

The  result  of  Theorem  3.3.2  produces  a  consistent  estimator 
of  the  mean. 


Proof : 

The  factors,  ir. ,  should  actually  be  considered  to  be  estimates  of  the 
probabilities,  and  will  be  therefore  indicated  as  in,  the  observed 
relative  frequency  of  events  occurring  in  the  interval  i  <  £  <  i+1. 
Since  the  coefficient  outside  the  surmation  is  a  deterministic  factor, 
as  are  the  (b)1^  terms,  vre  can  consider  the  estimate  of  the  mean  to 

A 

be  a  weighted  average  of  other  estimated  values,  namely  the  in's. 

Each  of  the  in  estimates  is  known  to  be  a  consistent  estimator  of  in, 
so  we  must  show  that  the  weighted  average  of  a  set  of  consistent  esti¬ 
mators,  is  itself  consistent.  This  can  be  easily  shewn  if  the  compon¬ 
ents  of  the  set  are  independent,  since  if  we  have  the  random  variables 
u,  v,  and  w  such  that: 

w  =  a  u  +  b  v 

then: 

E[w]  =  a  F[u]  +  b  E[v] 
and  assuming  independence  of  u  and  v, 


°w  *  a  5u  *  b 


Of  a  total  of  n  sanples,  nu  will  contribute  to  n  and  the 
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remaining  will  contribute  to  v,  in  such  a  manner  that  as  n  -*•  00 
both  and  will  also  increase  without  bound,  so  that: 


0 


which  proves  that  the  estimator  is  not  only  unbiased,  but  also  ap¬ 
proaches  the  true  value  of  the  population  mean  with  unity  probability, 
and  is  therefore  a  consistent  estimator. 


3. 3. 2. 3  Moment  Estimates  from  the  Approximated  f(x)  Function 

The  density  function  approximations  of  Section  3.2.3,  which 

A 

converted  rectangular  segments  of  <$>(£)  into  rescaled  rectangular 

A 

segments  of  f  (x) ,  can  also  be  utilized  for  moment  estimates.  Fran 
Equation  (3.30)  we  know  that  this  rescaling  function  is, 

PA  »  \  /[(b  -1)  (b)1] 

and  therefore,  moment  estimates  can  be  obtained  by: 


[xn] 


(3.65) 


where  the  caret  indicates  that  it  is  an  estimate  of  the  expected  value. 
Note  that  this  estimate  is  not  in  general  the  same  as  assuming  that  all 
of  the  probability  mass  is  concentrated  at  the  center  of  the  interval, 
which  generates  estimates  of: 
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(3.66) 


e2(x"|  -  £  ,(b-L|)-Mln(b-  i)(h)ip, 
i=0 

The  assumption  that  the  probability  is  uniformly  spread  across  each 
interval  is  typically  closer  to  the  actual  density  function  being 
rrodeled  than  is  the  concentrated  mass  assunption.  Hcwever ,  am  even 
better  density  approximation  can  be  formed  if  additional  information 
can  be  obtained  oonoeming  the  distribution  of  the  probability  within 
each  interval.  Such  information  is  often  available  in  the  relative 
probabilities  of  the  adjacent  intervals  as  shewn  in  Figure  3.3.1,  and 


Figure  3.3.1  A  First-Orbar  Approximation  to  the  Probability 
Distribution  Across  a  Histogram  Interval 


can  be  utilized  to  improve  the  moment  estimates  by  creating  a  first- 

order  approximation  to  the  slope  of  the  density  function  over  each  in- 

til 

terval.  Instead  of  assuming  that  the  density  is  uniform  over  the  i 
interval,  it  would  then  be  assured  to  be  li  ^arly  distributed  according 
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to  the  function: 


f^x)  =  nux  +  B^, 


fb)1  <  x  <  (b) 1+1 


(3.67) 


with 


fi  (interval  midpoint)  =  (3.68) 

The  probability  area  of  the  interval  is  maintained  automatically  for 
the  first-order  approximation,  as  indicated  in  Figure  3.3.1. 

The  analysis  of  such  first-order  corrections  is  fairly 
lengthy,  and  therefore  has  been  relegated  to  Appendix  D.  Ibis  analysis 
includes;  (1)  a  general  approach  to  the  problem,  (2)  the  selection  of 
appropriate  adjacent  slope  weighting  functions,  and  (3)  the  application 
of  the  method  to  both  uniform  and  log-histograms.  Seme  example  applica¬ 
tions  will  be  shown  in  Section  3. 3. 2. 4  to  indicate  the  effect  of  such 
a  first-order  correction,  and  to  compare  moment  estimates  fran  uniform 
and  log-histograms. 


3. 3. 2. 4  A  Comparison  of  Moment  Estimates  from  Uniform  and 

Log-Histograms 

A  series  of  Monte  Carlo  tests  were  made  to  evaluate  the 
moment  estimation  capabilities  of  uniform  and  log-histograms,  and  in¬ 
dicated  surprisingly  comparable  results  from  the  two  techniques.  The 
log-histograms  were  expected  to  fare  rather  poorly  in  such  a  comparison 
due  to  the  fact  that  the  log-scale  intervals  are  relatively  large  at  the 
upper  end  of  the  scale,  and  the  moment  estimates  are  quite  sensitive  to 
this  region.  However,  the  first-order  correction  techniques,  as  de¬ 
veloped  in  Section  3. 3. 2. 3,  were  found  to  be  quite  effective  in 
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reducing  the  errors  in  the  estimates,  and  were  applicable  to  a  variety 
of  density  function  shapes. 

The  tests  considered  five  different  densities;  the  uniform. 
Erlang (2),  exponential,  hyperexponential,  and  truncated  1/x  density 
functions.  Randan  variables  in  the  range  of  0  to  1  were  generated  by 
a  ccmputer  subroutine  and  were  transformed  to  cover  the  range  of  o  to 
500  with  the  desired  density  by  a  transformation  of  variable.  The 
resulting  randan  variables  were  sorted  into  uniform  and  log-histograms, 
with  10  intervals  each,  and  estimates  of  the  mean,  msansquare,  and 
variance  were  then  calculated  from  the  histogram  data.  Hie  actual  sam¬ 
ple  moments  were  also  accumulated  during  the  tests  and  were  utilized  to 
calculate  the  estimate  errors  for  each  test.  These  results  have  been 
summarized  in  Table  3.3.2,  but  are  presented  in  detail  in  Appendix  E 
along  with  a  more  complete  description  of  the  test  procedures. 

3.3.3  Seme  Guidelines  for  the  Measurement  of  Moments 

The  only  moments  of  interest  are  typically  the  mean,  mean- 
square,  and  variance,  so  we  shall  only  consider  these  three  moments, 
although  marry  of  the  considerations  and  limitations  could  apply  to 
higher  moments  as  well.  There  are  several  possible  alternatives  for 
gathering  moment  estimates  including; 

•  collecting  raw  data  samples 

•  calculating  the  moments  at  the  data  source 

•  oorpacting  the  data  into  histograms  or  other  grouped  data 
formats. 
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TABLE  3.3.2 

A  SUMMARIZATION  OF  TOE  MONTE  CARLO  TEST  RESULTS 


1 

< 

* 

Density 

Function 

l 

Sample 

Mean/ 

Variance 

Uniform  Histogram 
Error 

Log-Histogram 

Error 

Mean 

Variance 

Mean 

Variance 

Uniform 

1 

247 

21,300 

+0.3% 

+  7.0% 

+1.0% 

1 

+  2.8% 

Erlang  (2) 

82.0 

3,280 

-1.4% 

+20.3% 

-0.7% 

-35.0% 

Exponential 

80.7 

6,090 

+2.3% 

+  8.0% 

-0.6% 

-  5.4% 

Hyper¬ 

exponential 

80.8 

7,310 

+2.5% 

+  4.3% 

+0.3% 

-  4.8% 

Truncated 

Vx 

+9.0% 

-  4.0% 

+1.6% 

+  4.1% 

These  techniques  are  listed  in  order  of  their  preference  from 
a  statistical  viewpoint,  but  unfortunately  several  practical  constraints 
limit  the  applicability  of  the  first  two.  For  example,  to  calculate  a 
mean  value  to  within  2%  to  3%  accuracy  with  a  90%  confidence  level  re¬ 
quires  about  1000  samples,  and  such  large  samples  can  require  an  ex¬ 
cessive  amount  of  the  buffering  and  transmission  facilities  of  a  net¬ 
work.  In  contrast,  we  obtained  about  this  same  accuracy  in  the  Monte 
Carlo  tests  of  Section  3. 3. 2. 4,  but  only  transmitted  the  ten  histogram 
slot  values.  This  reduction  was  possible  due  to  the  compacting  of  the 
data  at  the  source.  Such  compaction  could  further  reduce  the  trans¬ 
mission  requirements  by  directly  computing  the  moments,  but  for  other 
than  the  mean,  the  squaring  and  multiple  precision  operations  (to  avoid 
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overflow)  are  too  tinv*  oonsuning  in  most  small  oormuni cations  control 
ocmputers. 

The  accumulated  statistics  measurement  facility  as  described 
in  Section  2.5.1  provides  sane  practical  examples  of  the  trade-offs  be¬ 
tween  these  n orient  estimating  techniques.  A  primary  performance  meas¬ 
ure  of  the  net  is  message  delay,  and  such  delays  are  recorded  by  running 
totals  of  messages  sent,  and  time  for  the  round  trip  (message  out  and 
KFNM  back) .  Higher  manent  data  would  have  been  very  desirable  rather 
than  having  only  the  average  values,  but  the  multiplicity  of  destina¬ 
tions  made  it  impractical  to  keep  separate  histograms  on  each,  and  a 
composite  histogram  would  be  virtually  meaningless.  The  selection  of 
an  appropriate  set  of  histogram  intervals  would  have  been  interesting 

i 

due  to  the  open  ended  range  of  delay  values,  and  the  lack  of  any  round 
trip  times  of  less  than  about  eight  milliseconds.  A  suitable  set  of 
histogram  intervals  might  have  been  obtained  by  dividing  all  delay  val¬ 
ues  by  eight  (a  simple  shift  operation)  and  considering  the  intervals 
to  be  0  to  8,  8  to  16,  16  to  32,  etc.  up  to  a  "catch-all"  interval 
such  as  ">  256".  However,  the  need  for  seven  such  intervals  for  -ach 
of  32  destinations  would  increase  the  measurement  load  considerably. 

No  precise  stabanent  can  be  made  regarding  the  accuracy 
requirement  for  an  open  ended  set  of  experiments,  although  seme  reason¬ 
ableness  guidelines  can  be  stated  based  on  the  natural  units  of  the 
system  such  as  the  packet  length,  the  time  to  service  an  average  mes¬ 
sage,  and  the  fixed  service  time  due  bo  the  packet  overhead.  Queueinq 
delays  will  be  of  the  order  of  one  to  five  times  the  average  service 
tine  (under  test  conditions  such  as  described  in  Chapter  4) ,  and 
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should  be  measured  with  a  resolution  of  at  least  10%  to  20%  of  this 
average  service  time.  Higher  resolution  is,  of  course,  very  desirable 
but  must  be  balanced  against  the  possibility  of  overflow  (for  short 
vrard  lengths)  or  the  need  for  excessive  measurement  data  transmission. 

The  allowable  error  in  the  second  moment  will  depend  on  the 
experiment  being  conducted,  but  in  the  queueing  delay  example ,  its 
effect  can  be  seen  in  the  Pollaczek-Khinchine  formula  (SA61) : 

0  ? 

Eftime  in  the  system]  =  x  +  (3.69) 

1  "  p  2x 


where: 


p  =  Xx 


For  a  traffic  intensity  of  about  p  equal  to  0.5  and  for  exponential¬ 
like  service  times,  the  two  terms  will  be  of  acrparable  magnitude,  and 
therefore  should  have  similar  accuracies.  If  we  consider  only  the  x 
and  x  error  effects,  and  write  the  estimates  as  the  actual  value 
plus  an  error  term,  we  have: 

E[x]  =  (1  +  e^x  (3.70a) 


E[x2]  =  (1  +  e2)x 
P  =  X  (1  +  e^x 


The  queueing  delay  term  of  Equation  (3.69)  then  becomes: 


/V  A  O 

P  .  E[x_! 
1  -  p  2E[x] 


X(1  +  e1)x  (1  +  e2)xz 
1  -  Ml  +  e^)x  2(1  +  e^)x 


(3.70b) 

(3.70c) 


(3.71) 
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or: 


(1  +  e\,)X  x' 


E  [queue  delay]  = 


2 [1  -  X(1  +  e1)x] 


For  snail  values  of  p  this  estimate  is  approximately, 


~2 

E  [queue  delay  |  small  p]  =  (1  + 


(3.72} 


(3.73) 


while  for  values  of  p  near  0.5,  the  estimate  becomes: 

a  __  1  +  ~Z 

E[queue  delay  |  p  ~  0.5]  -  * - — -  (X  x  ) 

1  "  el 

or  approximately: 

E [queue  delay  |  p  »  0.5]  =  (1  +  +  e^JX  x2  (3.74) 

Therefore,  the  fractional,  or  percentage  errors,  in  the  estimates  of  x 

...... 

and  x  cure  of  comparable  significance  in  determining  the  queueing 
delay  estimate  accuracies. 

The  variance  can  be  calculated  iron  these  moments  about  the 

origin  by: 

o2  =  E[x2]  -  (E[x])2  (3.75) 

and  similarly,  an  estimate  of  the  variance  can  be  obtained  from  esti¬ 
mates  of  these  moments  by: 

O2  =  E[x2]  -  (E [x] ) 2  (3.76) 

If  we  subtract  Equation  (3.75)  from  Equation  (3.76)  we  obtain  an  ex¬ 
pression  for  the  error  in  the  variance  estimate, 
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o  »2  7 

er(cr)  =  -  a 


or: 

er(o2)  =  {E[x2]  -  E[X2]}  -  { (E[x]  )2  -  (E[x))2}  (>.77) 

which  can  be  written  as: 

er(a2)  =  er(x2)  -  {(E[x]  +  E[x])  •  er(x)} 

For  most  purposes,  this  expression  can  be  approximated  by, 

2  ~2  -  - 

er(o  )  S  er(x  )  -  2  x  er(x)  (3.78) 

which  relates  the  error  in  the  variance  estimate  to  the  errors  in  the 
estimates  of  the  mean  and  meansquare  values.  Note  that  if  the  latter 
two  errors  are  of  the  same  sign,  they  vail  tend  to  cancel  in  the  vari¬ 
ance  estimate,  and  this  is  precisely  the  situation  that  occurred  in  the 
exponential  density  exanple  of  the  Monte  Carlo  test  described  in 
Section  3. 3. 2. 4  and  Appendix  E. 

The  square  root  of  the  variance  is  the  standard  deviation, 
which  is  dimensionally  the  sane  as  the  mean  and  indicates  the  spread  or 
dispersion  of  the  density  function.  Intuitively,  the  accuracy  of  the 
standard  deviation  should  be  ocrrparable  to  that  of  the  mean,  and  could 
be  represented  by, 

a  S  (1  +  Gl)o  (3.79) 

for  which  the  corresponding  error  in  the  variance  becomes: 

o2  =  (a)2  S  (1  +  2ex)02  (3.80) 
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Therefore,  while  the  errors  in  the  meansquare  and  mean  values  should  be 
ocrrparab'e,  the  allowable  error  in  the  variance  can  be  twice  this  value. 
This  relationship  can  also  be  seen  in  the  Pollaczek-Khinchine  formula 
of  Equation  (3.69)  which  can  be  written  as: 

Eftime  in  system]  =  x  +  [M2  +  °21  (3.81) 

and  clearly  shows  that  the  percentage  errors  in  the  mean  and  standard 
deviation  are  of  oornparable  significance.  The  moment  estimates  from 
the  Monte  Carlo  tests  of  Section  3. 3. 2. 4  indicate  the  second  moment 
estimates  from  histograms  will  be  somewhat  less  accurate  than  the  esti¬ 
mates  of  the  mean,  and  will  therefore  be  the  more  significant  source  of 
error  in  such  calculations. 

3.4  Artifact  Considerations  and  Corrections 

Artifact  in  measurement  data  refers  to  any  effect  that  results 
in  a  difference  between  the  actual  system  variables  (as  they  would 
exist  if  the  measurements  had  not  been  made) ,  and  the  observed  system 
variables  obtained  from  the  measurement.  Such  artifact  can  occur  either 
by  measurement  effects  which  change  the  actual  system  behavior,  such  as 
introducing  additional  delays,  or  from  effects  which  distort  the  meas¬ 
ured  results  without  actually  changing  the  system  behavior.  An  example 
of  the  latter  type  of  "apparent  artifact"  would  occur  if,  in  a  hardware 
rronitor ,  the  sampling  period  is  inadvertently  made  in  synchronism  with 
some  time  varying  behavior,  and  thereby  does  not  record  a  true  picture 
of  the  behavior.  Both  types  of  artifact  will  be  found  to  occur  in  the 
network  measurements  as  discussed  in  the  following  paragraphs . 
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There  are  two  basic  aspects  of  the  network  artifact;  (1)  that 
introduced  in  the  IMP  to  generate  the  statistics  data,  and  (2)  that 
introduced  in  the  transnission  facilities  to  get  the  measurement  re¬ 
sults  to  the  NMS.  We  shall  consider  each  of  these  areas  with  regard  to 
the  identification  of  the  artifact  conditions,  the  possible  ways  of 
correcting  for  the  artifact,  the  ways  in  which  the  artifact  was  reduced, 
and  sane  suggestions  on  how  subsequent  implementation  might  further 
reduce  the  artifact. 

3.4.1  The  Use  of  IMP  Resources  to  Generate  the  Measurements 

The  measurement  progrars  reside  within  the  IMP  core  memory 
and  compete  for  the  IMP  processing  resources  to  generate  the  statistics 
data.  Since  many  of  the  IMP  processing  functions  require  real-time  in¬ 
put  data  handling  (e.g. ,  setting  vp  a  new  buffer  before  the  next  packet 
starts  to  arrive)  the  neasurement  functions  have  been  designed  to  mini¬ 
mize  their  interference  with  the  input  operations.  Except  for  the 
trace  and  arrival  statistics,  the  measuronents  concentrate  on  the  modem 
channel  output  functions  to  record  the  packet  size  data,  etc.  (The 
HJST-IMP  data  transfers  do  not  have  such  time  constraints,  since  they 
are  bit  asynchronous,  and  therefore  measurement  data  are  taken  in  both 
directions  between  the  IMP  and  HDST.)  The  modem  output  test  data  also 
includes  the  effects  of  the  measurement  message  lengths,  which  will  be 
considered  as  part  of  transmission  artifact. 

The  snap-shot  statistics  are  intended  to  capture  the  state 
vector  of  the  IMP  queues  and  routing  tables,  but  have  been  assigned  a 
priority  below  the  interrupt  handling  functions  to  also  avoid 
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interference  with  the  real-time  routines.  Therefore,  the  higher  prior¬ 
ity  task  queues  will  always  be  empty  when  the  background  statistics 
program  is  executed,  and  similarly  the  background  program  may  be  inter¬ 
rupted  to  handle  subsequent  higher  priority  functions.  As  a  result,  the 
snap-shot  data  may  be  skewed  in  time,  and  will  not  necessarily  be  com¬ 
pletely  consistent.  For  example,  the  total  number  of  store-and- forward 
buffers  may  differ  from  that  reported  in  the  queue  length  measurements, 
if  a  lengthy  interruption  has  occurred  between  the  time  that  these  two 
items  cure  recorded.  Such  skated  results  have  been  observed ,  but  typi¬ 
cally  occur  only  when  very  heavy  traffic  is  introduced  into  the  IMP. 

The  need  for  core  memory  space  for  the  measurement  routines 
introduces  an  artifact  in  any  blocking  effect  measurements,  since  if 
that  same  space  had  been  utilized  for  buffers,  the  blocking  might  not 
have  occurred.  However,  this  effect  would  only  be  of  concern  if  block¬ 
ing  occurred  frequently,  and  otherwise  the  information  gained  on  the 
system  performance  is  well  worth  the  cost.  In  the  case  of  the  IMP, 
approximately  750  16-bit  words  are  utilized  for  the  neasurement  rou¬ 
tines,  which  would  correspond  to  about  ten  additional  buffers  beyond 
the  present  total  of  seventy  (HE70) . 

The  oembination  of  the  above  mentioned  decisions  and  design 
ccmpromises ,  effectively  reduced  the  IMP  resource  artifact  to  a  very 
small  effect.  Unfortunately,  the  transmission  artifact  is  not  nearly 
as  negligible,  and  requires  considerable  care  in  defining  experiments 
and  in  correcting  the  artifact  distortions  of  the  data. 
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3.4.2 


The  Use  of  the  Network  Transmission  Resources  for  the 
Measunanent  Data 


The  measurement  data  messages  tend  to  be  fairly  long  and 
occur  at  frequencies  which  can  utilize  considerable  transnission  band¬ 
width  as  shown  in  Table  3.4.1.  These  data  messages  have  been  ccnpreseed 


TABLE  3.4.1 

TRANSMISSION  REXJUIFENuNTS  FOR  MEASUREMENT  DATA 


Measurement 

Approximate  Data 
Message  Length 

Period 

Typ.  BW6 

Traces 

80  +  160  N  bits1 

as  generated 

0.4  Kb/sec.2 

Snap-Shots 

2200  bits 

0.82  sec. 

2.7  Kb/sec. 

Arrival  Times 

64  +  16  M  bits3 4 

t  4 

see  note 

0.34  Kb/sec.5 6 

Aocusn.  Data 

3120  bits 

12.8  sec. 

0.25  Kb/ser 

1.  N  is  the  number  of  trace  blocks  in  a  trace  message. 

2.  Based  on  N  =  2  and  sent  once  per  second. 

3.  M  is  the  number  of  arrival  times  recorded  per  message. 

4.  Sent  either  every  1.6  sec. ,  when  60  arrivals  occur,  or 
when  another  statistics  program  must  be  run,  whichever 
occurs  first. 

5.  Based  on  M  =  30  and  sent  every  1.6  seconds. 

6.  Peak  bandwidth  requirements  are  much  larger  due  to  the 
burst  nature  of  the  measurement  traffic. 


119 


in  size  as  much  as  possible  by  utilizing  the  IMP  CPU  to  compact  the 
data  into  counts,  averages,  histograms,  etc., but  there  are  a  large  vari¬ 
ety  of  IMP  functions  to  record  and  the  resulting  test  data  messages  are 
up  to  four  packets  in  length.  These  data  messages  can  affect  the  net¬ 
work  performance  by  the  extra  traffic  load,  longer  queues,  etc.,  and  in 
same  cases,  require  corrections  to  the  reoorded  values.  For  example, 
each  accunulated  statistics  message  includes  the  previously  sent  sta¬ 
tistics  message  in  the  modem  channel  traffic,  but  being  of  known  length, 
it  can  be  subtracted  out  to  correct  the  measurements.  However,  if  this 
was  one  of  several  messages  sent  to  the  MC,  the  round  trip  times 
could  only  be  partially  corrected  based  on  known  average  data  message 
delays. 

The  snap-shot  statistics  are  particularly  expensive  in  terms 
of  transmission  requirements  due  to  their  length  and  frequency,  but  are 
taken  at  rather  gross  time  increments  relative  to  the  "time  constant" 
of  the  network.  Seme  further  data  ocmpaction  within  the  IMP  might  be 
possible,  but  is  not  readily  definable  due  to  the  changing  measurement 
needs,  e.g.,  for  peak  queue  lengths  in  one  experiment  versus  average 
queue  lengths  in  another.  An  additional  artifact  involved  with  the' 
snap-shots  is  due  to  the  fact  that  they  are  taken  in  approximate  syn¬ 
chronism  across  cne  net,  and  therefore  introduce  traffic  in  bursts.  Of 
course,  the  snap-shots  are  only  generated  at  those  IMP's  which  have  had 
the  appropriate  flags  set  for  the  experiment.  (This  is  true  for  each 
of  the  different  measurement  facilities  as  described  in  Section  2.5.) 

The  methods  of  artifact  reduction  which  Estrin,  et  al.  des¬ 
cribed  (ES67A)  cure  applicable  candidate  techniques  if  one  were  to 
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reduce  the  transmission  artifact  of  the  network  measurements.  Those 


methods  are  paraphrased  below: 

1.  make  measurements  on  qross  events  rather  than  detailed 
operations. 

2.  selective  injection  of  measurement  "probes"  at  key  points 
of  interest. 

3.  use  of  external  measurement  devices  such  as  a  hardware 
monitor. 

The  first  technique  was  somewhat  orthogonal  to  the  measure¬ 
ment  intent  of  obtaining  detailed  measurement  data  to  provide  insight 
into  the  network  behavior ,  but  was  utilized  in  the  sense  that  compacted 
data  was  taxen  when  appropriate.  The  selectivity  of  the  aoond  tech¬ 
nique  has  varying  degrees,  and  in  the  case  of  the  network,  involved 
enabling  any  subset  of  the  five  h^sic  measurement  routines,  (i.e.,  the 
acomulated  statistics,  snap-shots,  traces,  arrival  times,  and  status 
reports)  at  the  IMP'r  involved  m  an  experiment.  Further  selectivity 
could  have  reduced  the  transmission  requirements  somewhat,  but  only  if 
a  clear-cut  subdivision  could  be  defined  within  the  data  sets.  The 
choice  of  having  fixed  or  variable  data  message  formats  v:as  decided  in 
favor  of  fixed  formats  due  to  the  ccrnpli cations  in  the  IMP  measurement 
code  if  it  were  made  variab?e.  However,  this  choice  would  have  to  be 
reconsidered  if  the  net  changed  radically  in  the  number  of  nodes  or 
their  interconnectivi  ty . 

The  third  method,  i.e.,  the  introduction  of  special  measure¬ 
ment  related  devices,  would  hove  considerable  significance  ir.  the 
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resulting  measurement  capabilities.  For  example,  the  addition  of  a 
magnetic  tape  recorder  at  each  IMP  could  provide  a  storage  facility  for 
measurement  data  that  v«ould  allow  more  thorough  measurements  and  would 
avoid  the  transmission  artifact  during  the  experiment.  The  data  would 
be  transmitted  after  the  test,  at  seme  off  hour,  or  by  mail  if  the 
quantity  of  data  was  very  large.  Several  interesting  tests  could  be 
run  if  such  equipnent  were  available,*  such  as  blocking  effect  measure¬ 
ments  which  are  presently  thwarted  by  the  blockage  itself.  Simultaneous 
measurements  could  also  be  taker,  at  more  nodes  of  the  network,  where 
presently  the  bandwidth  limitations  require  fairly  austere  experiment 
planning. 

No  attenpt  has  been  made  in  these  discussions  of  artifact  to 
identify  each  and  every  source  of  artifact,  but  merely  to  point  out 
several  typical  cases  in  which  such  artifact  occurs,  and  how  it  might 
be  corrected,  or  in  other  designs,  eliminated.  Each  experiment  must 
carefully  consider  the  artifact  which  is  introduced  for  that  combina¬ 
tion  of  traffic  and  measurement  data,  and  attenpt  to  correct,  or  at 
least  identify,  such  sources  of  error. 

3.5  Data  Analysis 

While  our  basic  measurement  philosophy  was  in  complete 
agreement  with  Harming’ s  observation  that  "the  purpose  of  measurement 
is  insight,  not  nutbers";  one  must  also  admit  that  the  direct  output  of 
measurement  is  numbers,  and  insight  must  cane  from  analysis  and 

★ 

HDST  cooperation  in  the  measurement  activity  could  provide  an 
alternative  source  of  local  storage. 
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reduction  of  these  nureric  results.  Unfortunately,  it  is  difficult  to 
develop  algorithms  for  providing  insight  fron  a  given  set  of  data 
values,  and  therefore  much  of  the  data  analysis  requires  hunan  inter¬ 
vention,  at  least  until  a  hypothesis  has  been  formulated  on  some  aspect 
of  the  network  behavior  of  interest.  Due  to  this  need  for  hunan  obser¬ 
vation  of  the  data,  our  experiments  have  been  run  utilizing  line  printer 
records  (at  the  UCLA  Network  Measurement  Center  HDST) ,  for  the  various 
measurement  data  fran  each  experiment. 

These  records  were  initially  printed  in  a  hexadecimal  outcut 
form,  and  shortly  thereafter  in  a  more  readable  decimal  form.  However, 
these  records  were  very  difficult  to  scan  for  particular  items  of  in¬ 
terest,  so  special  print  formats  were  developed  which  provided  several 
advantages  including  appropriate  annotation,  a  more  convenient  arrange¬ 
ment  of  the  data,  and  sere  simple  calculations  and  bar  charts.  Such 
print-outs  were  shown  in  the  figures  of  Section  2.5,  and  have  been 
found  to  be  quite  adequate  for  both  the  rapid  scanning  of  the  results 
during  the  data  collection  phase  of  an  experiment,  and  for  subsequent 
data  analysis  and  reduction. 

The  data  analysis  activities  will  oertainly  evolve  from  this 
initial  manual  mode  of  operation  to  more  sophisti  cated  computer-aided 
techniques.  The  first  step  of  this  change  is  presently  being  imple- 
irented  as  the  data  collection  routines  are  being  integrated  into  the 
UCLA  time-sharing  systan.  This  change  allows  the  experimenter  to 
analyze  his  data  as  it  is  obtained,  and  to  take  advantage  of  the  other 
system  facilities  for  this  purpose;  a  significant  improvement  over  the 
stand-alone  operation  in  which  the  Signa  7  computer  was  dedicated  to 
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the  data  collection  phase  of  experiments.  Future  epochs  ol  this 
evolution  will  include  the  utilization  of  other  network  resources  for 
data  reduction  retrieved,  and  display,  and  perhaps  may  also  result  in 
machine-aided  perception  or  insight  through  the  use  of  artificial 
intelligence  programs  in  the  network. 
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CHAPTER  4 


MEASUREMFNTS  OF  NETWORK  PERPORMAfJCE 

The  set  of  measurement  tools  described  in  Chapter  2  and  the 
analysis  techniques  of  Chapter  3  can  best  be  illustrated  by  actual  net¬ 
work  measurement  situations.  In  this  chapter  we  will  consider  some 
example  measurements  to  further  demonstrate  the  utility  of  the  measure¬ 
ments  (e.g. ,  to  simulate  a  prescribed  level  of  traffic  and  measure  the 
resulting  delays)  and  for  exploratory  tests  to  answer  selected  "what 
if"  questions  such  as,  "What  happens  to  the  routing  mechanism  if  the 
traffic  load  becomes  extremely  heavy?"  These  tests  will  therefore  be 
oriented  toward  determining  what  questions  should  be  asked  regarding 
the  network  behavior.  In  contrast,  the  measurements  which  will  be  con¬ 
sidered  in  Chapter  5  will  be  based  on  the  verification  of  analytic 
models  of  the  system  behavior.  Actually  these  two  measurement  func¬ 
tions  operate  together  in  an  iterative  manner,  with  the  exploratory 
measurements  resulting  in  the  formulation  of  models,  and  subsequent 
tests  being  designed  to  test  the  validity  of  the  models.  These  latter 
tests  will  typically  provide  same  additional  insights  which  can  be 
utilized  to  improve  the  model,  and  then  additional  verification  tests 
would  be  run,  etc. 

The  motivation  for  these  test  and  model  development  efforts 
is  to  gain  insight  into  the  system  behavior,  and  in  particular,  to 
determine  the  "sensitive"  areas,  i.e.,  those  areas  in  which  small  pa¬ 
rameter  variation  can  result  in  large  performance  changes.  The  intent 
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of  Chapters  4  and  5  is  to  provide  sane  exanples  of  this  measurement- 
modeling  technique  to  a  variety  of  problem  areas,  seme  of  which  will  be 
seen  to  have  definite  areas  of  parameter  sensitivity,  while  others  will 
be  relatively  insensitive  to  change.  Both  types  of  conclusions  provide 
useful  information  regarding  the  system  behavior,  although  the  former 
is  certainly  more  gratifying. 

4.1  Message  Delay  Measurements 

Message  delay  is  a  primary  performance  measure  in  a  computer 
network  due  to  its  contribution  to  the  overall  response  time  of  an  in¬ 
teractive  message,  although  like  "response  time",  it  is  subject  to  a 
variety  of  definitions.  FOr  example,  in  the  NPL  network  described  in 
Section  1.2,  the  aanponert  of  the  delay  due  to  the  network  was  defined 
as  being  the  time  from  the  receipt  of  the  last  bit  of  a  packet  at  the 
source  node,  to  the  start  of  the  output  transmission  at  the  destination. 
Variations  of  the  definition  are  possible,  such  as  by  considering  the 
latter  time  to  be  the  end  of  transmission  at  the  destination,  or  alter¬ 
natively,  to  consider  definitions  based  on  message  delays  rather  than 
packet  delays.  Most  such  definitions  differ  by  reasonably  constant 
terms,  and  therefore  can  be  compared  in  a  relatively  simple  manner  if  a 
precise  definition  of  each  measurement  is  given  in  terms  of  its  start¬ 
ing  and  ending  points  in  time.  A  potential  problem  arises  because 
these  starting  and  ending  events  occur  at  different  sites,  i.e.,  the 
source  and  destination  nodes.  Unless  each  site  has  an  accurate  and 
synchronized  clock,  the  time  difference  will  be  meaningless ,  and  while 
such  synchronization  technqiues  are  feasible,  the  ARPA  network  delay 
measurements  were  based  on  time  values  at  a  single  node  by  utilizing 
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the  notion  of  round  trip  times  of  messages  and  their  associated  RFNM's 
(Request  For  Next  Message).  The  component  delays  involved  in  this  defi¬ 
nition  are  shown  in  Figure  4.1.1  for  the  case  of  two  adjacent  nodes, 
with  more  separated  nodes  requiring  additional  store-and- forward  queue¬ 
ing  delays  and  transmission  times.  The  delays  of  the  RFNM  return  will 
be  small  due  to  their  short  leixgth,  and  reasonably  predictable  due  to 
their  high  priority,  so  that  the  return  delay  can  be  subtracted  frcm 
the  total  round  trip  time  to  give  a  good  estimate  of  the  message  trans¬ 
mission  delay.  Unless  otherwise  specified,  all  of  the  following  delay 
measurements  will  be  based  on  the  total  round  trip  delay. 

4.1.1  Delays  Due  to  Store-and-Forward  Handling 

The  example  utilized  in  Section  2.5.1  to  describe  the  accumu¬ 
lated  statistics  functions,  demonstrates  the  effect  of  additional  store- 
and- forward  operations  between  the  source  and  destination  sites.  How¬ 
ever,  that  particular  example  was  not  felt  to  reflect  the  normal  top¬ 
ology  because  of  the  unexpected  ratio  in  which  the  traffic  was  split 
between  the  paths  via  the  SRI  and  RAND  nodes.  The  test  was  repeated  at 
a  later  date  and  resulted  in  the  data  shewn  in  Table  4.1.1,  which  met 
the  expected  traffic  distribution  and  round  trip  delays.  Closer  in¬ 
spection  of  the  previous  test  data  indicated  that  the  BBN-MIT  channel 
was  inoperative  and  therefore  resulted  in  an  extra  store-and- forward 
delay  for  the  MIT,  Lincoln  Laboratories,  and  Case  destinations. 

The  round  trip  delays  consist  of  two  major  components;  (1) 
the  serial  transmission  requirements  of  the  message,  and  (2)  the  propa¬ 
gation  delays  which  occur  twice  in  the  round  trip  measurement.  The 
artificial  traffic  messages  being  sent  had  an  average  length  of 
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Figure  4.1.1  A  Detailed  Picture  of  the  Delays  Involved 
in  a  UCLA-RAND  Message  Transmission 


TABLE  4.1.1 


MEASURED  ROUND  TRIP  TRANSMISSION  TIMES 
FROM  UCLA  TO  SEVERAL  DIFFERENT  SITES 


Measured  Average 

No.  of  S  &  F  Round  Trip  Tirre  S  &  F  Path 
Destination  •  Transmissions  msec.  frcrn  UCIA 


UCSB 

Utah 

MIT 

Lincoln  Labs 
Case 


22.4  msec 
53.2  msec 
94.0  msec 
114.0  msec 
137.5  msec 


direct 
via  SRI 
via  RAND 
via  RAND 
I  via  RAND 


320  bits,  which  require  a  totcil  of  about  12.0  msec,  for  serial  trans¬ 
mission,  when  one  includes  the  overhead  characters  and  the  RFNM.  The 
propagation  delay  is  about  10  ysec./faile  (HE70)  ,*  resulting  in  a  cross¬ 
country  delay  of  approximate ly  30  msec.  Therefore,  the  round  trip  time 
fran  UCIA  to  MIT  would  be  expected  to  require  36  msec,  for  the  three 
s tore- and- f orward  transmissions  and  60  msec,  for  the  propagation  delays 
(both  ways) ,  for  a  toted  of  about  96  msec.  The  observed  round  trip 
delay  is  shown  in  Table  4.1.1  to  be  94  msec.  The  round  trip  time  to 
Case  Institute  of  Technology  is  the  largest  of  the  values  shewn  due  to 
the  five  store-and-forward  delays  and  the  fact  that  the  path  from  UCIA 
to  Case  goes  via  Boston,  which  adds  even  more  propagation  delay.  (Such 
seemingly  peculiar  topological  aspects  are  due  to  the  evolving  nature 
of  the  network,  with  only  a  subset  of  the  ocmrunication  net  being  pres¬ 
ently  installed.)  In  each  instance  the  delays  were  quite  acceptable 

As  compared  to  5.4  ysec./mile  for  the  speed  of  light  in  free  space. 
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from  a  user  interaction  point  of  view,  although  of  course  no  queueing 
delays  were  involved  in  these  transmissions  (due  to  the  lack  of  any 
carpeting  traffic) . 


4.1.2  Delay  as  a  Function  of  Message  Length 


Message  length  variations  influence  the  round  trip  times  in 
two  ways;  (1)  by  the  serial  transnission  delays,  and  (2)  by  the  effect 
of  the  message  service  requirement  on  the  queueing  delays.  This  latter 
ocnponent  of  delay  will  be  considered  in  Section  4.1.3,  and  for  new  we 
will  only  be  concerned  with  the  serial  transmission  delay.  In  the 
description  of  the  previous  experiment  this  component  of  delay  was 
approximately  12.0  msec,  for  a  320  bit  message,  of  which  6.4  msec,  is 
required  for  the  transmission  of  the  average  size  320  bit  message  (at 
20  usee. /bit  or  50  Kbits/sec.),  2.7  msec,  is  required  for  the  header 
and  line  control  characters,  and  an  additional  2.7  msec,  is  for  the 
RETJM.  These  values  do  not  add  to  12.0  msec.,  since  this  latter  figure 
includes  the  fact  that  the  message  lengths  are  random  variables,  and 
seme  messages  will  exceed  one  packet  in  length,  and  therefore  will  re¬ 
quire  additional  header  and  line  control  overhead,  with  the  expected 
number  of  packets  per  message  (for  the  exponential  message  length  case) 
being; 


E [#  pkts.  per  mes. ]  =  1/(1  -  e 


(4.1) 


where  Lg  is  the  segment  or  packet  length,  and  SL^  is  the  average 
message  length.  (This  equation  will  be  developed  later  in  Section 
5. 2. 3.1).  When  this  effect  is  considered,  we  obtain  the  expected 
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message  service  times  shown  in  Table  4.1.2.*  These  values  do  not 
include  the  RFNM  delay,  since  it  is  not  appropriate  for  some  of  our 
later  use  of  these  delay  ccrrponents ,  such  as  the  following  queueing 
delay  analysis. 


TABLE  4.1.2 

EXPECTED  SERVICE  TIMES  FOR  VARIOUS  LHK7TH 
PARAMETER  VALUES  OF  RANDOM  ARTIFICIAL  TRAFFIC  GENERATOR 


Avg.  Mes. 

Length  from 
Sigma  7  Gen. 

{ (32-bit  words) 

Avg.  Mes. 
Length  in 
the  IMP 
(bits) 

E[#  pkts] 
per  message 

E[*  bits  per 
mes.  incl. 
pkt.  overhead) 

E[mes. 
service 
time) 
(msec. ) 

1 

32 

1.00 

168 

3.4 

2 

64 

1.00 

200 

4.0 

5 

160 

1.00 

296 

5.9 

10 

320 

1.05 

463 

9.3 

15 

480 

1.14 

635 

12.7 

20 

640 

1.27 

813 

16.3 

25 

800 

1.40 

990 

19.7 

30 

960 

1.54 

i 

1170 

23.4 

4.1.3  The  Queueing  Component  of  Message  Delay 

If  several  message  sources  are  in  contention  for  the  service 
facility,  queues  may  form  with  resulting  increases  in  the  average  mes¬ 
sage  delay.  Such  queueing  behavior  is  a  function  of  the  average 
message  service  requirement,  x,  and  the  average  arrival  rate  of 


For  exanple,  Table  4.1.2  indicates  an  average  message  service  time  of 
9.3  msec,  for  a  message  with  an  expected  length  of  320  bits.  If  we 
add  the  2.7  msec,  service  for  the  RFNM,  we  obtain  the  12.0  msec,  delay 
value  used  in  the  above  discussion. 
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messages,  X,  with  these  two  factors  determining  the  server  utilization 
parameter,  p,  by  the  relationship: 

p  =  X  x  (4.2) 

The  message  service  requirement  is  a  function  of  the  message  length  as 
described  earlier,  and  the  average  arrival  rate  can  be  controlled  for 
experimental  purposes  by  the  selection  of  the  two  other  parameters  of 
the  artificial  traffic  generator,  i.e.,  the  "transmission  attempt  in¬ 
terval",  T  ,  and  the  number  of  message  generators  being  utilized.  (The 
KETCM  mechanism  for  congestion  control  is  the  reason  for  considering  T 

a 

to  be  the  time  interval  between  "attempts"  at  transmission,  since  no 
transmission  will  occur  on  a  link  which  is  wai  J.ng  for  a  RFNM  return.) 

The  RENM  will  also  create  a  second  aspect  of  concern,  since 
each  RFTJM  must  be  acknowledged.  Therefore,  a  number  of  ACK's  will  be 
in  contention  for  the  service  facility  along  with  the  messages  them¬ 
selves,  and  in  heavy  traffic  conditions,  will  effectively  increase  each 
service  time  by  the  3.0  msec,  that  is  required  to  transmit  a  higher 
priority  ACK.  However,  for  lesser  traffic  levels,  the  ACK's  can  not  be 
considered  to  be  simply  an  additional  portion  of  the  average  service 
time,  but  instead,  appear  to  be  separate  high  priority  arrivals.  These 
two  effects  can  be  combined  by  use  of  the  priority  model  result  that 
will  be  developed  in  Section  ”  l.l.  This  result  shews  that  the  queue¬ 
ing  delay  for  a  priority  message  (the  ACK  in  this  case)  will  be: 

Wx  =  V(1  "  aXx^)  (4.3) 

where  Wq  is  the  expected  time  to  complete  a  service,  aX  is  the 
average  arrival  rate  of  the  ACK's,  and  x^  is  the  service  time  of  the 
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ACK's.  Similarly,  the  expected  queueing  delay  of  non-priority  arrivals 
(the  regular  messages  in  this  case)  is  shewn  to  be: 


w2  =  ”1/(1  -  P) 


(4.4) 


where  the  value  of  p  is  a  function  of  the  oonposite  arrival  rate  and 
service  times  (i.e. ,  messages  and  ACK's).  By  substitution  of  Equation 
(4.3)  into  Equation  (4.4)  and  reoognizing  that  the  arrival  rate  of 
messages  and  ACK's  will  be  equal,  we  obtain: 


w2  =  a  -  yca>  <1  -  yy 


(4.5) 


where  X  is  the  service  time  for  an  ACK,  and  x  is  the  average 
a  *- 


composite  service  time  for  a  message  and  an  acknowledgement , 


xc  =  Xa  + 


(4.6) 


The  value  of  can  be  shewn  to  be: 


wo  =  v?/2 


(4.7) 


where  the  theoretical  value  of  the  average  arrivax  rate  is: 


(4.8) 


with  N  being  the  number  of  generators  utilized  in  the  experiment.  We 
also  need  an  expression  for  ,  which  must  include  both  the  ACK's  and 
the  messages.  Since  the  two  sets  of  arrivals  are  equally  likely,  one 
can  show  by  Equation  (A. 6)  from  Appendix  A,  that: 


^^(Xa)2+i[(Xa): 


+  2  Xa  \  +  2 


<v2] 


(4.9) 
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which  simplifies  to: 


'V2  +  Xa  \  +  <V2 


(4.10) 


Containing  these  equations  produces  an  expression  for  the  expected 


message  queueing  delay  of: 


N 

2T 


(xj‘ 

a 


+  x. 


(\)2 


a 


1, .  lil 

• 

,  N<Xa  +  V 

l  Ta  J 

r  Ta  J 

(4.11) 


which  is  plotted  as  the  theoretical  queueing  delay  curve  in  Figure 
4.1.2.  Each  of  the  test  conditions  used  to  obtain  the  plotted  experi¬ 
mental  data  involved  constant  values  of  N/Ta  and  X^,  with  the 
variable  being  the  average  message  service  requirement,  x^.  The 
figure  shows  that  a  rather  poor  match  is  obtained  between  the  theory 
and  experiment  for  the  four  generator  case,  with  this  discrepancy  beinq 
due  to  the  FFNM  effect  which  increases  the  message  inter  arrival  times.* 
The  other  two  test  cases  utilized  a  larger  number  of  generators,  which 
decreased  this  RFNM  effect,  and  provided  a  noch  better  agreement  with 
the  theoretical  expectations.  Hcwever,  these  two  cases  also  exhibit  a 
departure  fran  the  theoretical  values  for  larger  message  lengths,  and 
other  test  data  confinred  that  this  behavior  was  also  due  to  the  RFNM 

effect. 

Figure  4.1.3  shows  a  related  effect  that  was  observed  in  the 
experiments ,  i.e.,  that  the  effective  value  of  p  decreases  due  to  the 


*Each  message  generator  utilizes  a  separate  logical  link  which  is 
"blocked"  following  a  transnission  until  such  time  as  the  RFNM  is 


returned. 
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THEORETICAL  Q  CKL  AV  f  *  >*  Pen 


AVERAGE  MESSAGE  LENGTH  (BYTES) 


TRAFFIC  INIlUfV^ 


1.6 


1.4 


AVERAGE  MES  LENGTH  -  40  BYTES 
L_  x  ■  12.3  msec 


1.2 


N  i 
200 


/ 


N 


1 200  msec 

'  NO.  OF  GENERATORS 


/ 


^effective 

y  VARIATION 
DUE  TO  ALT 
ROUTING 
EFFECTS 


REGION  IN  WHICH  p 
CAN  BE  SIMULATED 
RELIABLY 


5  10  16  20 

NUMBER  OF  GENERATORS  BEING  UTILIZED 


25 


Figure  4.1.3  Theoretical  and  Measured  Value*  of  p  for  the  fi,  jficial  Traffic  Generator 
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increased  interarrival  times.  In  fact,  the  system  approaches  a  "closed 
queueing  system"  in  which  the  "customers"  circulate  through  the  server 
and  then  return  for  more  service  after  seme  random  idle  period.  For 
such  a  system  the  arrived  rate  must  asymptotically  approach  the  service 
rate,  i.e.,  p  approaches  unity  as  shown  in  the  figure.  The  effect  of 
alternate  routing  perturbs  this  portion  of  the  curve  since  it  provides 
an  instantaneous  increase  in  the  service  rate,  and  makes  the  region 
rather  unpredictable .  However,  the  actual  region  of  interest  is  the 
lower  portion  of  the  curve  which  approximates  the  more  classical  randan 
arrival,  infinite  population,  queueing  model.  This  data  will  be  util¬ 
ized  to  select  the  experiment  parameters,  i.e.,  message  length,  number 
of  generators,  and  the  "transmission  attempt  interval",  T  ,  for  sub- 

a 

sequent  experiments  such  as  the  priority  model  verification  experiment 
of  Section  5. 1.2.5. 

4.2  Measurements  Related  to  Network  Applications 

Applications  of  the  network  were  initially  limited  by  the 
lack  of  an  established  protocol  for  HOST- to- HOST  transmission,  although 
several  sites  made  use  of  the  network  by  establishing  special  purpose 
protocol  subsets  for  handling  their  particular  functions  of  interest 
such  as  interactive  work,  file  transmissions,  or  graphics  display  func¬ 
tions.  Rather  than  summarizing  the  measurement  facilities  relative  to 
a  number  of  these  application  areas,  we  will  present  a  more  detailed 
accounting  of  the  measurements  that  were  taken  on  the  activity  of  one 
set  of  users. 
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4.2.1 


Measurements  of  Network  Usage 


The  activity  of  interest  involved  the  use  of  the  DEC  PDP-10 
at  the  University  of  Utah  and  the  XDS  940  at  SRI,  with  the  general 
nature  of  the  interaction  being  as  follows.  A  file  would  be  sent  frcxn 
SRI  to  Utah,  and  then  a  number  of  interactive  debug  operations  would  be 
performed,  with  transmission  from  the  XDS  940  to  the  PDP-10  being  on  a 
character-by-character  basis,  and  responses  being  handled  on  a  line  at 
a  time  basis.  The  files  were  transmitted  in  blocks  of  4608  bits,  which 
was  a  convenient  multiple  of  the  24-bit  word  length  of  the  XDS  940  and 
the  36-bit  word  length  of  the  PDP-10,  and  also  provided  the  desired 
block  size  of  128  words  for  the  PDP-10. 

The  established  protocol  involved  several  special  characters 
to  indicate  the  message  type  and  length,  such  that  a  single  character 
message  required  five  IMP -words  of  transmitted  data.  Due  to  these 
character-by-character  and  file  transmission  protocol  conventions,  al- 
nost  all  of  the  SRI-to-Utah  transmissions  consisted  of  either  5-word 
messages  or  5-packet  messages  respectively,  as  shewn  in  the  sample 
print-out  of  Table  4.2.1.  Each  line  corresponds  to  one  12.8  second 
statistics  message,  with  the  time-of-day  being  determined  by  adding 
12.8  second  incranents  to  the  starting  time  since  the  data  values  were 
taken  consecutively.  The  sequence  of  events  which  is  shewn  includes  a 
10-word  "file  name"  message,  the  file  transnission  of  67  5-packet 
messages  plus  a  4— packet  file  fragment,  an  "end  of  file  message,  and 
a  sequence  of  interactive  traffic.  This  particular  file  transmission 
consisted  of  approximately  39,300  bytes  and  occurred  over  a  25.6  second 
period  for  am  average  bandwidth  usage  of  about  12.5  btoits/seoond.  The 
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TABIjE  4.2.1 

A  SUBSET  OF  THE  ACCUMUIATED  STATISTICS  MFASUREJET 
OF  HE  SRI-UTAH  TRAFFIC 
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best  available  copy, 
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*  'EASURED  IN  TERMS  OF  ET  WDRDS 

**  *tEASURID  IN  TERfIS  OF  THE  NUMBER  OF  PACKETS  PER  MESSAGE 


maximum  file  transmission  bandwidth  observed  during  any  12.8  second 
interval  was  one  in  which  73  5-packet  messages  were  transmitted.  Each 
message  was  the  standard  576  byte  reoord,  for  a  total  bandwidth  usage 
of: 

26.3  Kb/sec  for  data  ("effective"  data  rate") 

0.3  Kb/sec  for  narking,  padding,  and  protocol  format 
3.9  Kb/sec  for  network  header,  checksum,  etc. 

30.5  Kb/sec  total  bandwidth  used. 

Transnission  occurred  during  the  entire  interval  since  the 
file  transmission  was  measured  over  three  12.8  sec.  intervals ,  and  this 
was  the  center  interval .  The  average  round  trip  time  for  a  message  was 
159  msec,  (measured  from  the  time  the  source  IMP  receives  the  first 
packet  until  it  receives  the  RFNM) .  The  minimum  round  trip  time  would 
be  about  138  msec,  based  on  the  time  for  serial  transmissions  and  propa¬ 
gation  delays.  The  extra  delay  of  159  minus  138,  or  21  msec. ,  was  due 
to  the  delays  in  the  PDP-10  acceptance  of  messages.  The  effective 
round  trip  cycle  must  include  an  additional  12  msec,  for  HOST  processing 
of  the  KETIM  and  for  the  leader  and  first  packet  transmission  fran  the 
HOST  to  the  IMP.  Therefore,  the  maximum  single  link  data  rate  is  ap¬ 
proximately  one  576  byte  reoord  every  150  msec,  or  30.7  Kb/sec.  of 
actual  data,  ooctpared  with  the  observed  maximum  effective  data  rate  of 
26.3  Kb/sec.  A  similar  calculation  using  the  measured  average  round 
trip  time  and  the  resulting  effective  data  rate  indicates  that  about  4 
to  5  msec,  are  required  for  HOST  processing  of  the  RFNM  and  setting  up 
the  next  transnission.  These  results  are  surrrnarized  be  lew: 
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Measured 


Theoretical  mininum 
(or  maximum) 


Round  trip  time 

Effective  data  rate 

Average  HOST  RFNM 
Proc.  &  mes. 
set  up  time 

HOST  delay  in  taking 
mes. 


159  msec. 
26.3  Kb/sec. 
5  msec. 

21  msec. 


138  msec.  min. 
30.7  Kb/sec.  max. 


The  round  trip  time  required  for  messages  between  the  two 
HOST'S  was  plotted  in  Figure  4.2.1  as  a  function  of  message  length  L. 
Each  point  of  this  scatter  plot  represents  one  interactive  message, 
i.e.,  a  one-packet  message.  The  scattering  is  an  indication  of  the 
frequency  of  occurrence  of  both  message  length  variations  and  round 
trip  delay  variations.  A  definite  lower  limit  of  delay  was  observed  as 
indicated  by  the  dashed  line,  and  this  limiting  delay  function  is  just 
what  would  be  expected  based  on  estimated  values  of  the  various  acm- 
panents  of  delay.  In  this  minimum  delay  case,  there  is  no  queueing 


time  such  that  the  expected  delay  consists  of: 

•  Serial  transmission  at  source  IMP  =  2.0  +  0.32L  msec. 

•  Propagation  and  modem  delay  =7.5  msec. 

•  Processing  time  at  destination  =0.5  msec. 

•  Serial  IMP-HOST  transmission  =  0.3  +  0.16L  msec. 

•  Serial  transmission  of  RFNM  =  2.0  msec. 

•  Propagation  and  modem  delay  =7.5  msec. 

•  Processing  of  RFNM  ~  0.2  msec. 

Total  20.0  +  0.48Lmsec. 


This  estimated  total  delay  agrees  very  well  with  tie  experimental 
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Figure  4.2.1  Round-Trip  Delay  Measurements  of  the  SRI-Utah  Traffic 


minimum  delay  values  as  shown  in  Figure  4.2.1.  The  other  points  in  the 
figure  represent  messages  whose  delay  values  exceeded  the  minimum  due 
to  queueing  delays  and/or  HOST  delays  in  taking  the  message  from  the 
IMP. 

4.2.2  Estimating  User  Behavior 

The  example  of  the  SRI-Utah  interactive  traffic  measurements 
can  also  serve  as  an  indication  of  the  degree  to  which  one  can  obtain 
information  regarding  the  user  behavior.  Table  4.2.1  shewed  a  pattern 
of  data  transfers  which  could  readily  be  seen  to  be  a  sequence  of  file 
transfers  followed  by  bursts  of  interactive  traffic.  Of  course,  no  in¬ 
formation  could  be  obtained  regarding  the  usage  of  the  files,  i.e. , 
they  could  have  been  display  images  or  small  data  bases,  but  the  fact 
that  they  all  consisted  of  five  packet  messages  gives  some  information 
regarding  the  file  transfer  protocol.  Similarly,  the  predominance  of 
five  word  messages  might  lead  one  to  speculate  that  the  user- to- com¬ 
puter  messages  were  sent  on  a  character-by-character  basis,  while  the 
computer  response  traffic  was  sent  in  longer  and  more  variable  length 
messages.  (The  latter  were  included  in  the  measurements,  but  are  not 
shown  in  the  table.) 

4.2.3  Estimating  Traffic  Statistics 

The  reduction  of  the  data  from  the  SRI-Utah  traffic  example 
would  have  been  made  more  difficult  if  the  traffic  had  been  a  composite 
representation  of  several  simultaneous  user  transnissions ,  but  seme  in¬ 
sights  could  probably  have  still  been  gained.  The  initial  tests  of 
user  behavior  were  typically  measurements  of  single  user  characteristics 
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due  to  the  absence  of  heavy  network  usage,  but  as  the  traffic  loads 
build  up,  such  cases  will  occur  with  less  frequency.  However,  in  many 
instances  the  ccrposite  data  transnission  may  be  of  particular  interest, 
such  as  the  case  of  a  HOST  site  desiring  to  measure  its  total  network 
activity  over  a  given  period  of  time.  A  hypothetical  example  of  such 
data  is  shown  in  Figure  4.2.2  and  is  the  result  of  artifically  generated 
traffic.  This  particular  exarple  was  selected  at  randan  fran  a  se¬ 
quence  of  test  data  involving  messages  with  various  average  lengths, 
but  in  each  case  having  an  exponential  distribution  of  lengths.  The 
histogram  data  values  were  utilized  (with  the  correction  formulas  as 
developed  in  Appendix  D)  to  determine  the  mean  message  length,  with  an 
average  length  of  65.6  IMP -words  being  obtained  ccnpared  to  a  theoreti¬ 
cal  expected  value  of  64.0  words.  (The  effect  of  the  correction  was  to 
reduce  the  error  fran  3.2  to  1.6  words.)  A  second  example  is  shewn  in 
Figure  4.2.3,  with  an  estimated  mean  value  of  25.9  oorpared  to  a  theo¬ 
retical  value  of  24.0.  In  this  example  the  correction  factor  reduced 
the  error  from  5.0  to  1.9.  This  exarple  also  shows  the  peak  of  the  log- 
histogram  density  function  occurring  at  the  mean  of  the  exponential 
function;  a  result  that  was  developed  in  Section  3.2.2. 

Figure  4.2.3  also  shows  the  density  estimate  from  the  log- 
histogram  data  corrpared  with  the  theoretical  density  function,  and  a 
reasonably  good  match  is  seen.  In  aontrast,  the  data  from  Figure  4.2.2 
would  not  have  a  very  recognizable  density  shape  due  to  the  small 
sanple  size,  although  such  a  plot  has  not  been  shown. 

A  sanple  histogram  for  a  modem  output  channel  is  also  shewn 
in  Figure  4.2.2,  and  serves  as  an  example  of  a  potential  source  of 
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error  for  the  channel  estimates.  The  histogram  interval  covering  the 
range  of  32  to  63  includes  a  large  number  of  full  63-word  packets, 
which  should  be  considered  separately  in  any  estiinates  made  from  the 
histogram.  In  the  example  we  can  correct  for  this  effect,  since  the 
HOST-to-IMP  data  shows  that  there  were  35  full  packets  (i.e. ,  19  from 
the  2-packet  messages,  10  from  the  3-packet  messages,  and  6  from  the 
4-packet  messages).*  Using  this  information,  the  estimated  length  was 
39.5  words  ocmpared  to  a  theoretical  value  of  40.4,  with  the  correction 
factor  causing  the  error  to  increase  from  0.8  to  0.9.  The  fact  that 
this  "correction"  actually  made  the  error  scmewhat  larger  is  a  reminder 
that  the  correction  will  improve  the  estimates  on  the  average,  but  not 
necessarily  in  each  individual  case. 

We  were  able  to  determine  the  number  of  full  63-word  packets 
on  the  modem  channel  for  the  above  example  because  those  packets  origi¬ 
nated  at  the  local  HOST  aid  additional  length  data  was  available.  In 
general,  we  can  only  make  such  a  calculation  based  on  long  term  average 
values ,  and  this  limitation  indicates  a  weakness  in  the  channel  data 
gathering  facilities.  An  additional  histogram  interval  for  full  63-word 
packets  would  be  a  worthwhile  extension  of  the  measurement  capabilities 
to  avoid  this  uncertainty . 

4.3  Interference  Between  Users 

A  major  concern  in  the  evaluation  of  a  computer- communications 
network  is  the  determination  of  how  many  simultaneous  users  the  network 
can  support.  Several  criteria  for  such  an  evaluation  are  possible, 

*Each  3-packet  message  contributes  two  full  packets ,  so  that  the  five 
3-packet  messages  result  in  ten  full  packets,  etc. 
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ranging  from  being  able  to  staisfy  the  peak  requirement  of  all  users 
simultaneously,  to  satisfying  the  average  need  for  all  users.  In  the 
latter  case,  if  a  user  required  an  average  of  500  bits/sec. ,  then  one 
might  estimate  that  a  50  Wait/sec.  transnission  facility  could  support 
100  such  users.  Of  course,  such  a  system  would  not  be  ,-orkable  since 
the  service  rate  barely  meets  the  average  demand  for  service,  and  very 
large  queues  and  delays  would  develop.  Ihe  other  case  is  overly  con¬ 
servative,  since  the  probability  of  all  users  requiring  their  peak 
service  simultaneously  is  very  small,  but  these  two  conditions  do  tend 
to  bound  the  number  of  users  that  such  a  network  could  supoort.  In  the 
following  analysis  we  shall  attempt  to  determine  a  better  measure  of 
the  network  capabilities  in  this  regard. 

4.3.1  Estimates  of  the  Nurber  of  Users  That  a  Ccmuni cations 

Channel  Can  Support 

Ihe  nature  of  the  measured  traffic  in  the  SRI -Utah  test  of 
Section  4.2.1  involved  both  file  transmissions  and  interactive  mes¬ 
sages,  with  the  two  types  of  traffic  being  seen  to  utilize  the  oemuni- 
cations  bandwidth  in  grossly  differing  ways.  The  file  transmissions 
were  quite  infrequent,  but  utilized  a  large  portion  of  the  available 
bandwidth  during  bursts  of  activity.  In  contrast,  the  interactive 
usage  consisted  of  longer  intervals  of  lew  bandwidth  transfers  inter¬ 
spersed  between  idle  or  "think"  periods.  Due  to  these  differences  in 
band-width  characteristics,  and  the  differing  consequences  of  delays  for 
the  two  classes  of  traffic,  one  should  not  expect  to  utilize  the  same 
criteria  for  estimating  the  number  of  simultaneous  users  that  could 
share  the  system.  Therefore,  we  shall  consider  the  two  aspects  of  the 
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problem  separately,  and  then  attempt  to  contoine  the  resulting  estimates. 

The  interactive  traffic  utilization  of  the  ccrrrrunication 
facility  for  several  simultaneous  users  is  analogous  to  the  time-sharing 
of  a  computer  system,  and  we  shall  take  advantage  of  these  earlier  re¬ 
sults  to  estimate  the  effects  of  sharing  the  communication  channel. 
Scherr  (9C67A)  developed  a  time-sharing  model  which  shewed  that  the  de- 
gradat’ c  i  of  response  due  to  nutliple  users  was  lew  until  the  number  of 
users  exceeded  a  threshold  or  saturation  point,  above  which  the  degrada¬ 
tion  rapidly  increased.  (Kleinrock  (KL69C)  generalized  this  model  to 
evaluate  multiple  channels  and  investigated  the  notion  of  saturation.) 
The  model  considered  exponential  think  time  and  service  cycles  for  the 
various  users,  with  an  average  think  time  of  6  seconds  and  an  average 
service  requirement  of  1/pC  seconds  per  service  cycle,  where  C  is 
the  rate  at  which  work  can  be  performed  by  the  server ,  and  1/p  -s  the 
average  service  requirement  of  an  arrival .  Hcwever,  due  to  the  sharing 
of  the  server,  each  user  spends  an  average  of  T  seconds  in  the  server 
during  a  oenpute  cycle,  and  then  returns  to  the  think  state  where  he 
spends  an  average  of  0  seconds  before  making  another  service  request. 
For  a  total  of  M  simultaneous  users,  the  number  in  the  think  state 
will,  on  the  average,  be  equal  to: 


M 


6 


t  +  e 


(4.12) 


which  will  produce  an  effective  arrival  rate  for  the  server  of 
approximately , 

=  M/(T  +  6) 
in 
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Hie  output  rate  of  the  server  will  be: 


\ 


out 


=  uC(l  -  po). 


(4.14) 


where  pC  is  the  service  rate  and  pQ  is  the  probability  that  the 
server  is  idle.  The  input  and  output  rates  must  be  equal,  such  that: 


PC(1  -  pQ)  =  IV  (T  +  6) 


(4.15) 


which  when  solved  for  a  normalized  value  of  T  becomes: 


pCT 


(4.16) 


Since  T  is  the  average  time  spent  in  the  prooessor  to  gain  1/yC 
seconds  of  service,  the  value  of  yCT  is  a  measure  of  the  system  de¬ 
gradation  due  to  the  M  users.  For  very  snail  values  of  M,  the 
degradation  factor  is  near  unity,  but  above  a  threshold  or  saturation 
value,  the  degradation  increases  rapidly  and  becomes  approximately 
proportional  to  the  number  of  users.  This  threshold  value  can  be  shewn 
to  be  (KL69C) : 

M'  =  1  +  pCe  (4.17) 

which  can  be  written  as: 

M'  =  (4.18) 

T 


where  t  is  the  average  service  time  requirement, 

x  =  1/yC  (4.19) 

Therefore,  the  relation  becomes, 

(M*  -  1)t  =  6  (4.20) 
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which  is  intuitively  satisfying  since  it  says  that  the  degradation 
which  any  user  will  see  will  begin  to  beccme  excessive  when  the  average 
service  requirement  of  the  other  M'  -  1  users  equals  his  average 
think  time. 

To  apply  this  theory  to  our  shared  oarmuni cation  channel 
model,  we  need  only  define  the  serial  server  by: 

C  =  transmission  rate  in  bits/sec. 

1/u  -  service  requirement  in  bits/message 
such  that  the  average  service  requirement,  t,  is: 

t  =  1/uC  sec.  per  message  (4.21) 

The  model  assumes  that  this  is  the  average  service  needed  considering 
both  user-to-conputer  and  computer- to- user  messages,  which  is  a  reason¬ 
able  assumption  for  full  duplex  lines  with  users  at  both  sites. 

The  measured  data  from  the  SRI-Utah  test  showed  an  average 
message  length  of  about  seven  16-bit  words  for  interactive  messages, 
which  including  the  header,  line  control  characters,  the  RFT3M,  and  the 
ACK's  results  in  an  average  total  service  requirement  of: 

x  =16.4  msec. /message 

The  think  time  was  not  measurable  with  the  accumulated  statistics  due 
to  their  12.8  second  period,  so  the  average  value  of  4.3  seconds  ob¬ 
served  by  Jackson  and  Stubbs  (JA69)  was  utilized,  resulting  in: 

6  =  4.3  seconds 

If  we  substitute  these  values  into  Equation  (4.18),  we  obtain: 

M'  =  -  -  1  =  260  users. 

T 
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However,  the  message  lengths  in  the  observed  activity  were  much  shorter 
than  would  be  expected  for  a  more  typical  mix  of  users ,  and  if  we  take 
Jackson  and  Stubbs  average  values  of : 

E[user-to-ocrputer  message]  =  12  characters 
E [cotrputer-to-user  message]  =  154  characters 

and  assume  an  equal  fraction  of  each  type  of  message,  we  obtain  an 
average  service  requirement ,  including  the  overhead ,  of: 

t  =  24.7  msec. /Message 

For  the  same  4.3  second  average  think  time,  the  critical  number  of 
users  becomes: 

M'  =  170  users. 

The  effect  of  the  file  transfer  bandwidth  requirements  is  much  more 
stringent  on  the  number  of  users  that  oould  be  supported  simultaneously. 
A  typical  file  length  for  the  SRI -Utah  test  was  about  25  K  bytes,  \hich 
for  an  effective  channel  bandwidth  of  about  40  Kbits/sec.  ,*  would  re¬ 
quire  5  seconds  of  channel  time  for  serial  transmission.  (The  actual 
transmission  would  typically  require  considerably  longer  due  to  HOST 
delays  in  accepting  and  inserting  messages.)  The  SRI-Utah  file  trans¬ 
missions  occurred  at  a  frequency  of  about  one  every  12  minutes,  during 
the  period  of  active  use,  such  that  one  user  would  require  on  the  order 
of  1%  of  the  effective  channel  bandwidth.  This  would  set  an  upper  limit 
of  about  100  such  users  before  exhausting  the  available  bandwidth,  but 
★  ' 

Based  on  a  traismission  overhead  due  to  header,  line  control,  etc.  of 
about  6  msec,  for  each  20  msec,  of  effective  data  transmission  on  the 
50  Ifoits/sec.  line. 
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for  delay  considerations,  the  maximum  would  be  limited  to  about  half 
that  number. 

We  have  therefore  established  some  rather  crude  estimates  of 
the  number  of  users  that  such  a  network  interconnection  oould  support 
as  being  about  170  interactive  lasers,  or  50  file  transmission  users. 
Since  users  typically  do  seme  of  each,  we  would  like  to  combine  these 
values  in  some  proper  manner.  A  reasonable  combination  would  seen  to 
be: 

#  interactive  users  A  #  file  transmit  users  , 

170  50  { 

which  satisfies  the  two  end  conditions,  and  allocates  the  bandwidth 
appropriately  for  a  mix  of  the  two  classes  of  users. 

4.3.2  Interference  Tests  Utilizing  Artificial  Traffic 

The  effect  of  multiple  users  can  be  simulated  by  utilizing 
many  artificial  traffic  generators  with  randan  arrivals,  but  this  situa¬ 
tion  differs  from  Scherr's  model  in  the  "think  time"  behavior.  In  his 
model,  the  think  time  period  started  at  the  oonpletion  of  the  previous 
service  period,  while  in  the  artificial  traffic  generation  scheme,  the 
interval  for  the  next  "transmission  attempt",  T  ,  is  selected  at  the 
time  the  message  is  enqueued  for  transmission.  As  a  result,  the  effec¬ 
tive  "think  time"  is  the  difference  between  T  and  the  time  spent 
obtaining  service.  We  can  include  this  effect  in  Kleinrock"s  equation 
for  the  number  of  users  at  the  "interference  break  point"  from  Equation 
(4.20); 

(M'  -  l)x  =  6 
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where  t  is  the  average  service  requirement  of  a  message  (an  average 
round  tnp  time  in  our  exanple) ,  and  8  is  the  think  time.  Tf  we  add 
t  to  both  sides  of  the  equation  we  obtain: 

M't  =  6  +  t  (4.23) 

in  which  6  +  x  is  our  T  ,  such  that: 

cl 

M't  =  T  (4.24) 

a 

For  the  examples  shown  in  Figure  4.3.1,  the  value  of  is  200  msec, 
and  the  average  round  trip  time  can  be  seen  to  be  about  20  msec. ,  re¬ 
sulting  in  a  value  of  M'  of  10,  which  agrees  well  with  the  experi¬ 
mental  data  for  both  exponential  and  fixed  message  lengths. 

The  theoretical  model  curves  shewn  in  the  figure  refer  to  the 
analysis  of  Equation  (4.11)  which  includes  the  effect  of  the  acknowl¬ 
edgement  traffic.  A  particularly  good  match  was  obtained  with  the  ex¬ 
ponential  message  length  experiment,  although,  as  expected,  the  two 
results  diverged  at  a  higher  number  of  generators  since  the  model  does 
not  include  the  effect  of  Figure  4.1.3  which  shews  that  p  gradually 
approaches  unity.  A  less  satisfactory  match  between  theory  and  experi- 
nrant  was  obtained  for  the  fixed  length  message  experiment. 

The  linear  slopes  of  the  two  experiment  cases,  i.e. ,  with 
exponential  (random  variable)  lengths  and  with  fixed  message  lengths, 
were  found  to  be  approximately  10  msec. /generator  and  7.5  msec. /genera¬ 
tor  respectively,  although  each  has  a  mean  service  time  of  12 . ->  msec, 
including  the  ACK  for  the  RFNM.  This  difference  is  academically  of  in¬ 
terest  and  will  be  pursued  in  a  later  investigation,  but  the  linear 
slope  portion  of  the  curves  is  not  of  particular  concern  for  our  present 
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Figure  4.3.1  Average  Round  Trip  Delay  as  a  Function  of  the 
Number  of  Message  Generators  Being  Utilized 
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needs  of  (1)  finding  the  point  at  which  interference  occurs,  and  (2) 
establishing  a  predictable  set  of  system  stimuli  for  use  in  subsequent 
experiments. 

4.3.3  Queueing  Behavior  in  a  Closed  System 

The  ultimate  user  interference  occurs  in  a  closed  cr  cyclic 
queueing  system  in  which  each  user  rejoins  the  queue  after  obtaining 
sane  amount  of  service.  Such  closed  systems  have  been  considered  by 
Gordon  and  Newell  (0067) ,  and  more  recently  by  a  nurrber  of  other  queue¬ 
ing  system  analysts.  Our  model  differs  fran  theirs  in  several  aspects, 
including  the  effect  of  non-zero  arrival  time,  i.e.,  a  message  arrival 
occurs  when  the  last  bit  of  a  serial  bit-stiing  has  been  received. 

Such  arrival  processes  have  been  described  by  Cohen  (0070) ,  who  also 
considered  the  effect  of  finite  storage  capacities. 

The  non-zero  arrival  time  functions  can  be  utilized  with 
unfinished  work  diagrams  as  shewn  in  Figure  4.3.2,  in  which  several 
different  arrival  rates  are  shown  (relative  to  the  service  rate) .  In 
each  exarrple,  three  'inits  share  the  server  in  a  cyclic  manner,  but  re¬ 
quire  a  time  interval  of  ST  to  rejoin  the  queue,  where  T  is  the 
fixed  service  tine.  In  fact,  an  additional  "arrival  queue"  is  implicit 
in  this  schene  and  corresponds  to  the  HDST-IMP  serial  interface  which 
may  transmit  only  one  message  at  a  time,  and  which  is  in  cascade  with 
the  IMP-IMP  queue,  with  the  latter  being  considered  to  be  the  actual 
service  facility.  In  all  three  examples,  the  unfinished  work  builds  up 
to  a  limiting  value  of  approximately  2T,  and  in  Figure  4.3.3,  this 
limiting  value  is  shown  to  be  (N  -  1)T  for  the  general  case  of  N 
items  in  the  cyclic  queue  (with  constant  service  times) .  As  can  be 
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Figure  4.3.2  Arrival  and  Unfinished  Work  Diagrams  for  a  Set  of  R  FNM  Driven  Message  Generators 
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Figure  4.3.3  Determination  of  the  Limiting  Queue  Level  for  a  Set  of  N  RFNM  Driven  Message  Generators 


seen  in  the  previous  case  for  N  equal  to  three,  this  limit  is  only  an 

approximation ,  but  the  peak  values  of  the  unfinished  work  function  can 

be  shown  to  range  between  the  value  of  N*T  for  8  equal  to  zero  and 

(N  -  1)*T  for  8  equal  to  unity,  which  provides  a  reasonably  tight 

bound  for  the  larger  values  of  N  which  are  us’xally  of  interest. 

Figure  4.3.3  indicates  a  geometric  proof  that  this  limiting 

value  of  unfinished  work  will  be  approximately  (N  -  1)*  T,  by  shewing 

that  the  rate  at  which  the  work  increases  is  (1  -  8)/B  until  such 

% 

time  as  a  message  generator  is  expected  to  provide  an  arrived,  but  is 

blocked  by  not  having  conpleted  its  previous  service.  In  the  figure, 

St 

the  i  generator  is  available  to  provide  an  arrival ,  but  the  i+1 
is  not,  and  results  in  the  steady-state  condition  of  the  unfinished 
work  diagram.  Knowing  the  slope  at  which  the  unfinished  work  accumu¬ 
lates,  and  the  fact  that  for  the  critical  timing  condition,  (N  -  1)  BT 

th 

seconds  occur  between  the  previous  arrival  from  the  i  generator  and 
the  time  at  which  the  next  arrival  should  s„art  across  the  HOST- IMP 
interface,  then  the  height  of  the  unfinished  work  can  be  seen  to  be: 

unfinished  work  =  (N  -  1)  (1  -  B)T  +  (N  -  1)BT 

which  simplifies  to: 

unfinished  work  =  (N  -  1)T.  (4.25) 

This  result  was  experimentally  verified  by  the  use  of  artificial 
traffic  with  various  numbers  of  generators.  For  example ,  snap-shot 
measurements  were  utilized  to  measure  the  number  of  messages  in  the 
system  for  the  case  of  ten  traffic  generators,  and  43  of  a  total  of  60 
measurements  indicated  a  value  of  nine,  with  eight  other  measurements 
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recording  a  length  of  ten  messages  and  with  the  other  nine  samples 
being  irrelevant  due  to  alternate  routing. 

A  similar  application  of  the  unfinished  work  model  can  be 
utilized  to  show  that  for  the  case  of  finite  buffering,  the  limiting 
value  of  the  unfinished  work  will  be  approximately  B  -  1  where  B  is 
the  number  of  available  buffers.  Intuitively,  this  result  says  that 
one  buffer  must  be  allocated  to  the  arriving  message  which  is  not  in¬ 
cluded  in  the  unfinished  work  until  its  arrival  is  ocrnplete. 

Cyclic  queueing  behavior  is  quite  artificial  in  the  normal 
network  environment,  but  does  occur  in  oer  -ain  test  conditions  such  as 
the  thru-put  experiments  which  we  describe  in  the  following  section. 

4.4  Ihru-Put  Tests  (RFTvM  Driven) 

The  camiunication  lines  between  the  IMP's  have  a  bandwidth  of 
50  Kbits/seoond ,  which  establishes  an  upper  bound  on  the  single  path 
thru-put.  However,  the  overhead  associated  with  each  message  limits 
the  total  thru-put  to  sane  lesser  value  as  shown  in  Figure  4.4.1.  The 
same  figure  also  indicates  the  measured  thru-put  for  the  case  of  alter¬ 
nate  routing  which  exceeds  the  50  Kbit/second  value  due  to  the  bifurca¬ 
tion  of  the  traffic  flow.  In  all  cases  the  messages  were  sent  at  the 
RFNM  driven  rate,  i.e. ,  as  soon  as  a  link  would  became  unblocked. 

Two  different  message  sizes  were  included  in  the  above  test. 
In  the  first  case,  the  messages  were  one  full  packet  in  length  so  that 
the  effective  packet  overhead  was  minimized  (relative  to  the  packet 
size) .  The  maximum  thru-put  for  such  messages  was  measured  to  be  38.6 
Kbits/seoond,  which  agrees  quite  well  with  the  theoretical  value  of 
39,0  Kbits/second  (based  on  20.0  msec,  of  useful  data  transfer  and 
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Figure  4.4.1  Thru-Put  Measurements  Between  UCLA  and  the  Neighboring  UCSB  IMP 


5.7  msec,  for  the  packet  overhead  and  the  acknowledgement  for  the  RFNM) . 
This  maximum  thru-put  is  rot  reached  for  the  case  of  a  single  link  due 
to  the  quiescent  period  while  waiting  for  the  RFNM  to  return.  This 
delay  is  about  13.5  msec.,*  which  adds  to  the  "non-productive" 


Based  on  a  measured  round  trip  time  frcm  UCLA  to  UCSB  of  22.8  msec, 
for  a  320  bit  message  (fran  Table  4.1.1)  and  the  kncwn  service  time  of 
9.3  msec,  for  a  message  of  that  length. 
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transmission  period,  and  results  in  an  expected  single  link  thru-put  of 
25.5  Kbits/seoond.  (The  measured  value  was  22.2  Kbits/seoond . ) 

The  thru-put  for  the  one-packet  message  case  can  be  seen  to 
increase  abruptly  when  the  transition  is  made  frcrrt  six  to  seven  traffic 
generators,  due  to  alternate  routing  via  the  SRI  path.  Alternate  rout¬ 
ing  occurs  at  this  number  of  generators  because  the  delay  values  in  the 
routing  table  are  numerically  equal  to  the  existing  queue  length  (at  the 
time  of  the  routing  update)  plus  four  times  the  HDP  number,  i.e. ,  the 
nurber  of  s tore- and- f orwand  "hops"  along  the  path.  Equation  (4.25) 
showed  that  the  expected  number  of  packets  in  the  "system"  is  N  -  1 
where  N  is  the  nunber  of  generators  (for  single  packet  messages). 

Since  the  number  in  the  queue  is  one  less  than  the  nunber  in  the 
"system"  (one  message  is  being  serviced) ,  the  routing  will  change  when 
N  -  2  exceeds  four,  i.e.,  for  N  equal  to  seven  as  observed  in  the 
measurement  data. 

The  increase  in  thru-put  for  the  seventh  generator  was  meas¬ 
ured  to  be  8.1  Kbits/seoond,  which  is  due  to  the  fact  that  approximately 
five  messages  were  enqueued  for  service  when  the  routing  was  changed. 
Therefore,  for  a  period  of  time,  each  channel  will  be  transmitting  with 
an  effective  instantaneous  data  rate  of  approximately  39  Kbits/seoond. 
This  parallel  transmission  will  last  until  the  five  or  six  messages  are 
served  (five  in  the  queue  plus  one  in  tie  server) ,  with  an  expected 
time  to  complete  the  service  of  about  125  msec.  With  routing  updates  of 
about  every  640  msec. ,  the  "old  routing"  channel  will  contribute  an 
added  data  rate  of  about: 

(125/640)  *39  Kbits/sec.  =  7.6  Kbits/sec.  (4.26) 
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which  is  approximately  equal  to  the  measured  increase  of  8.1  K/bits/ 
second. 

The  single  link  thru-put  for  the  full  8-packet  message  case 
was  seen  to  be  approximately  equal  to  the  saturation  value  of  the  single 
packet  case,  i.e. ,  38.5  Kbits/second.  This  equality  is  due  to  the  can¬ 
celing  effects  of  fewer  RFNM's  to  acknowledge,  and  the  short  quiescent 
delay  for  the  single  link  RFNM.  No  alternate  routing  was  observed  until 
the  two  generator  test,  above  which  the  traffic  split  evenly  between 
the  direct  UCIA-UCSB  and  the  UCEA-SRI-UCSB  routes. 

The  saturation  effect  for  the  8-packet  message  case  occurred 
at  a  lower  bandwidth  than  was  expected,  since  the  direct  and  alternate 
route  paths  should  each  be  able  to  deliver  messages  with  an  effective 
rate  of  about  43.5  Kbits/seoond  (which  is  higher  than  the  single-packet 
thru-put  since  there  are  fewer  RFNM' s  to  acknowledge).  The  measured 
limiting  value  of  about  70  Kbits/second  was  therefore  investigated 
further  to  try  to  determine  the  cause  of  this  behavior.  Several  pos¬ 
sible  reasons  for  this  discrepancy  were  considered  and  described  be  lav, 
since  they  represent  the  possible  limiting  factors  in  such  transmissions 
and  indicate  the  manner  in  which  the  statistics  gathering  tools  cat  re 
utilized  to  determine  the  cause  of  anomalies. 

a.  Lack  of  store-and-forward  buffers:  In  order  to  obtain  the 
maximum  thru-nut,  there  must  be  a  sufficient  quantity  of  store-and- 
forward  buffers  to  be  able  to  create  a  queue  that  is  long  enough  to 
keep  t..e  channel  busy  during  the  entire  routing  update  interval  which  was 
thought  to  be  about  520  msec.,  so  with  a  packet  service  time  of  23.0 
msec. ,  the  queue  would  have  to  build  up  to  a  total  of  about  23  buffers. 
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Since  we  were  told  that  there  was  a  total  of  25  store-and- forward 
buffers,  this  did  not  appear  to  be  the  limiting  factor.  Snap-shot 
measurements  also  shewed  that  typically  18  to  20  store-and-forward  buf¬ 
fers  were  in  use,  which  seemed  to  substantiate  this  conclusion. 

b.  Lack  of  reassembly  buffers:  Snap-shot  data  at  the  UCSB 
destination  IMP  shewed  that  typically  either  8  or  16  buffers  were  in 
use,  with  a  maximum  of  24  being  observed.*  Since  up  to  32  buffers  are 
available  for  reassembly,  this  was  not  the  problem.  Hie  lack  of  any 
queue  build  up  for  the  discard  "fake  HOST"  was  another  indication  that 
no  problem  existed  in  this  area. 

c.  Loss  of  bandwidth  due  to  routing  loops:  In  a  previous 
test,  a  considerable  loss  of  effective  bandwidth  was  noted  due  to  loops 
in  the  routing  (to  be  discussed  in  Section  4.5).  However,  the  accumu¬ 
lated  statistics  at  both  UCLA  and  UCSB  showed  that  the  transmissions 
were  all  properly  accounted  for,  i.e.,  no  extra  transmissions  due  to 
routing  loops  were  seen. 

c.  Loss  of  bandwidth  due  to  retransmissions :  Many  of  the 
accumulated  statistics  data  messages  indicated  transmission  errors, 
with  as  many  as  six  retransnissions  occurring  in  a  12.8  second  interval. 
However,  these  retransmissions  required  an  insignificant  bandwidth  as 
evidenced  by  the  thru-put  measurements.  (About  800-900  packets  would 
be  sent  in  one  12.8  second  period,  so  that  a  few  retransmissions  had 
little  effect.) 

e.  Loss  of  bandwidth  for  a  multi-HDP  path:  Ihe  alternate 
routing  path  includes  an  extra  store-and  forward  delay,  but  this  delay 

★ 

Reassanbly  buffers  are  presently  allocated  in  8-packet  blocks. 
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should  not  affect  the  thru-put  since  the  channel  is  in  effect  a 
pipeline,  am  additive  delays  do  not  affect  the  thru-put  of  such  a 
system. 

f.  Bandwidth  limitations  of  the  IDST-IMP  interface:  The 
HOST- IMP  interface  has  a  bandwidth  of  100  Kbits/second ,  which  can 
readily  handle  the  two  50  Kbit/second  channels  since  in  the  latter  case 
there  is  an  extra  overhead  due  to  the  packet  headers  and  line  control 
characters.  The  interface  is  full  duplex  (i.e.,  data  can  pass  in  each 
direction  simultaneously) ,  so  there  is  no  conflict  with  the  measurement 
data  being  collected. 

g.  Speed  limitations  of  the  Sigma  7  artificial  traffic 
generator:  The  Sigma  7  computer  must  perform  several  functions  includ¬ 
ing;  (1)  sending  the  artificial  traffic,  (2)  collecting  the  statistics, 
and  (3)  driving  the  line  printer  (the  output  device  for  the  statistics 
messages) .  Therefore,  there  was  concern  that  the  CPU  was  limited  in 
its  ability  to  generate  the  artificial  traffic.  This  was  shewn  not  to 
be  the  case,  by  transmitting  to  the  discard  "fake  HOST"  at  the  UCLA 
IMP  (instead  of  the  similar  discard  facility  at  UCSB.)  The  measured 
thru-put  for  the  local  discard  test  was  91.5  Kbits/second,  which  veri¬ 
fied  the  capability  of  the  Sigma  7  to  generate  the  necessary  bandwidth. 

h .  Trace  measurements  to  determine  the  detailed  behavior: 
Since  the  accumulated  statistics  and  the  snap-shots  both  indicated 
that  the  limitation  had  to  be  in  the  UCLA  IMP,  a  detailed  investigation 
of  the  packet  handling  behavior  was  made  by  tracing  all  of  the  packets 
at  the  UCLA  IMP  for  a  brief  instant  of  time.  (At  the  traffic  rates 
involved,  about  100  packets/second  were  generated  and  each  packet 
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resulted  in  a  separate  trace  message.)  This  trace  data  indicated  two 
errors  in  the  values  of  the  IMP  parameters  that  had  been  used  in  the 
calculations  of  part  (a)  above.  First  the  routing  updates  were  found 
to  occur  at  approximately  640  msec,  intervals  instead  of  the  520  msec, 
that  this  parameter  was  previously  understood  to  be,  and  secondly,  the 
upper  limit  on  the  store-and-forward  buffers  on  an  output  queue  must  be 
approximately  20  (instead  of  25)  ,*  since  the  interarrival  times  of 
packets  were  observed  to  increase  from  a  value  of  10.5  msec,  to  11.5 
msec,  (the  rate  at  which  the  two  channels  can  just  keep  up  with  the 
arrivals).  This  condition  will  force  the  queue  lengths  to  grow  to 
their  upper  bound,  and  neither  the  snap-shot  measurements  nor  the  trace 
data  showed  an  excess  of  20  store-and-forward  buffers  in  use.  If  we  re¬ 
consider  the  effect  of  the  finite  buffering  with  these  new  parameter 
values,  we  find  that  the  first  channel  will  have  approximately  18  pack¬ 
ets  to  serve  when  the  routing  change  occurs,  and  at  23.0  msec,  of  ser¬ 
vice  time  per  packet,  that  channel  will  remain  busy  for  about  420 
msec. ^  Since  the  next  routing  uodate  will  occur  a1; ter  640  msec.,  the 
first  channel  will  be  inactive  for  220  of  the  640  msec. ,  for  an  effec¬ 
tive  thru-put  contribution  of: 

420 

A  thru-pur  =  •  (43.5  Kbits/sec. )  =  28.5  Kbits/sec.  (4.27) 

for  a  total  estimated  thru-put  of  72  Etoits/seoond,  compared  to  the 

*A  subsequent  cneck  with  BEN  shewed  that  the  actual  routing  urdate 
interval  is  655  msec.  The  upper  limit  on  store-and-forward  Duffers  is 
25  (octal)  which  is  21  (decimal) . 

Trie  other  two  store-and-forward  packets  are  assumed  to  be  required  for 
the  SENT  queue. 
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measured  70  Kbits/second,  which  is  a  reasonable  match  between  the  theory 
and  measurement. 

This  test  activity  was  described  in  some  detail  to  indicate 
hew  the  various  measurement  tools  can  be  utilized  to  reooncile  differ¬ 
ences  between  the  expected  and  experimental  data  values.  However,  it 
also  points  out  an  equally  important  aspect  of  the  experimentation; 
namely  that  those  people  involved  in  the  network  measurement  and  model¬ 
ing  efforts  must  be  aware  of  any  changes  in  the  network  parameters  if 
they  are  to  be  able  to  properly  interpret  the  test  results. 

4.5  Alternate  Routing  Tests 

The  alternate  routing  mechanism  of  the  IMP  allows  it  to 
select  a  new  path  for  traffic  when  congestion  on  a  given  modem  channel 
exceeds  a  specified  limit.  However,  any  adaptive  control  system  may 
became  unstable,  and  in  the  case  of  the  network,  this  instability  is  in 
the  form  of  routing  loops  which  can  cause  messages  to  circulate  within 
the  net  rather  than  being  routed  towards  their  proper  destination. 

Such  loops  do  not  have  to  occur  endlessly  to  be  of  concern,  since  oven 
transient  loops  add  delay  to  the  transmission  due  to  the  longer  souroe- 
to-aestination  path,  and  by  effectively  increasing  the  traffic  load 
within  the  net. 

4.5.1  Detection  of  loops  in  the  Routing  Procedure 

Several  interesting  loops  were  observed  in  a  test  tnat  was 
designed  to  check  the  alternate  routing  mechanism  for  UCIA  to  RAND 
message  flow.  This  ccmuni cation  line  was  of  marginal  quality  during 
the  test,  and  numerous  retransmissions  were  required  on  the  direct  UCLA 
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bo  RAND  path.  The  combination  of  the  retransmissions  and  the  heavy 
traffic  load  lead  to  alternate  routing  both  via  the  SRI  node  and  via 
the  UCSB  node,  and  numerous  routing  loops  were  noted  from  the  total 
packet  counts  on  each  channel  as  indicated  in  Figure  4.5.1.  Most  of 


ARTI FICIAL  TRAFFIC:  70  32-BIT  WORDS  PER  MESSAGE 
REGEN.  PERIOD  -  50  MSEC 
5  LINKS  ACTIVE 


Figure  4.5.1  Loops  in  the  Routing  of  Packets  During  a  Special 
Test  Condition  (Not  Typical  of  Network  Flow) 
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the  alternate  routing  was  via  UCSB,  which  is  quite  strange  since  it  is 
a  longer  path  than  the  route  through  SRI. 

Accumulated  data  was  only  taken  at  the  UCLA  node,  but  the 
count  of  acknowledgements  sent  on  each  channel  were  good  indications  of 
the  number  of  packets  which  had  been  received  from  SRI  and  UCSB,  and 
these  were  counted  as  loops.  In  a  test  in  which  one  expected  loops, 
traces  would  also  be  taken  to  more  closely  measure  the  actual  looping 
behavior,  but  unfortunately  these  loops  were  discovered  well  after  the 
test  had  been  run. 

The  contoination  of  the  repair  of  the  UCLA  to  RAND  ocmnunica- 
tions  line  and  a  subsequent  BBN  rewrite  of  the  IMP  system  program,  re¬ 
sulted  in  completely  differing  results  when  the  test  was  repeated  at  a 
later  date.  This  latter  test  shewed  no  looping,  and  all  alternate 
routing  went  via  the  more  direct  SRI  path.  The  best  in  each  case  con¬ 
sisted  of  sending  messages  with  a  length  of  140  IMP-words  on  five  links 
with  a  regeneration  period  of  50  msec.  That  is,  every  50  msec,  the 
artificial  traffic  generator  would  send  a  message  on  any  or  all  of  five 

links  which  were  unblocked  (had  received  the  RFNM  fran  the  previous 

» 

transmission) .  This  network  loading  amounts  to  an  effect  data  trans¬ 
mission  rate  of  about  50  Kbits/seoond. 

We  should  stress  the  fact  that  the  loops  shown  in  Figure  4.5.1 
are  quite  unusual,  and  are  certainly  not  typical  of  what  one  would  ex¬ 
pect  to  see  in  the  network  flow.  However,  they  are  indicative  of  the 
type  of  loops  which  can  occur  in  a  store- and- forward  oermini cation 
system,  if  the  routing  is  not  well  behaved. 
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4.5.2 


Delay  Reduction  Due  to  Alternate  Routing 

Figure  4.3.1  showed  a  linear  increase  in  delay  as  additional 
message  generators  were  introduced  at  high  traffic  levels.  Hcwever , 
the  alternate  routing  effect  will  eventually  lessen  this  increase  due 
to  the  bifurcation  of  the  traffic  flew  into  two  or  more  channels,  as 
shown  in  Figure  4.5.2.  This  same  effect  was  not  observed  in  the  previ¬ 
ous  figure  (and  its  related  experiment)  since  the  shortest  alternate 
path  between  UCLA  and  RAND  involves  three  HOP'S,  i.e.,  UCTA-SRI- 
Standord-RAND.  Therefore  reasonably  large  queues  must  build  up  (of  the 
order  of  10  or  more  packets  enqueued) ,  before  alternate  routing  occurs, 
and  when  such  alternate  routing  does  occur,  the  delay  is  not  reduced 
significantly  due  to  the  three  store- and- forward  transmissions.  A  third 
difference  between  the  test  conditions  of  Figures  4.3.1  and  4.5.2 
involves  the  rate  at  which  messages  are  transmitted.  The  UdA-RAND 
test  utilized  randan  interarrival  times  with  an  average  time  between 
transmission  attempts  of  200  msec,  for  each  generator.  In  contrast,  the 
UCIA-UCSB  experiment  utilized  RITIM  driven  traffic ,  i.e.,  messages  were 
sent  upon  the  receipt  of  the  RFNM  from  the  previous  transmission. 

Two  other  aspects  of  the  tests  are  interesting  to  canpare. 

Ihe  UCIA-UCSB  exanple  of  Figure  4.5.2  has  a  delay  slope  of  about  28 
msec,  per  active  generator,  which  is  approximately  equal  to  the  26  msec, 
service  time  of  a  full  packet  message  and  the  ACK  for  the  RFNM.  The 
number  of  generators  at  which  the  interference  begins  to  occur  is  seen 
to  be  about  2  generators,  while  with  essentially  zero  "think  time  ,  the 
model  would  predict  a  value  of  1.  This  difference  is  believed  to  be 
due  to  the  "pipeline"  effect  of  the  system  such  that  two  message  bit 
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AVERAGE  ROUND-TRIP  DELAY.  MSEC 


streams  can  simultaneously  be  in  various  stages  of  the  transmission 
propagation,  and  RFNM  return  portions  of  the  system. 


UC'.A-TOUCSB  TRAFFIC 
(FULL  PACKET  MESSAGES) 

RFNM  DRIVEN 


DELAY  REDUCTION  \ 
DUE  TO  ALT  ROUTING 


4  6 

NO.  OF  ACTIVE  MESSAGE  GENERATORS 


Figure  4.5.2  Average  Round-Trip  Delay  as  a  Function  of  the  Number  of 
Active  Message  Generators  for  Full  Packet  Messages 
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CHAPTER  5 


MEASUREMENTS  AS  RELATED  TO  ANALYTICAL  MODELS 

One  of  the  primary  functions  of  the  network  measurement  effort 
is  to  provide  a  feedback  mechanism  for  the  development  of  analytic  mod¬ 
els.  For  example,  a  model  governing  some  aspect  of  the  IMP  behavior 
might  be  hypothesized  and  subsequently  either  verified,  corrected,  or 
bounded  by  experimental  test  caset,.  On  other  instances,  the  tests  may 
provide  insights  into  the  actual  IMP  behavior  which  would  suggest  ex¬ 
tensions  or  refinements  to  the  models,  and  hopefully  this  process  could 
be  used  iteratively  to  develop  models  which  satisfactorily  resemble  the 
actual  operation  of  the  subsystem  being  investigated. 

We  have  attempted  to  develop  a  set  of  test  cases  to  demon¬ 
strate  the  usage  of  measurements  in  providing  insights  and  guidance 
into  the  development  of  models  for  a  variety  of  different  aspects  of 
network  behavior.  Although  these  models  were  developed  as  a  part  of 
this  study,  they  have,  in  some  cases,  been  found  to  be  quite  similar  to 
previously  developed  models  of  other  application  areas.  Hcwever,  our 
intent  is  primarily  to  utilize  the  models ,  both  as  a  vehicle  to  demon¬ 
strate  the  measurement  feedback  process,  and  as  approxiir^v  .nns  to  the 
system  behavior.  These  earlier  references  are  cited  in  areas  which  were 
found  to  be  "reinvented",  with  the  other  areas  being  felt  to  be  original 
contributions . 


Preceding  page  blank 
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5.1 


A  Priority  Traffic  Analysis  Model 


A  cannon  practice  in  message  switching  systems  is  to  have  a 
priority  structure  built  into  the  queueing  system.  These  priorities 
may  be  based  on  either  service  requirements,  the  relative  urgency  of 
messages,  the  need  for  interactive  response,  or  any  other  appropriate 
rank  ordering  of  message  classes.  However,  we  will  assume  that  the 
priority  class  of  a  message  is  either  defined  by  the  message  source  or 
can  be  determined  algorithmically  upon  injection  of  the  message  into 
the  network,  and  further  that  the  priority  classification  of  a  message 
remains  fixed  as  it  travels  through  the  net. 

Before  defining  our  priority  model,  we  will  consider  a  wider 
scope  of  queue  and  service  disciplines  to  indicate  sane  of  the  alterna¬ 
tive  designs  that  might  be  considered,  and  to  put  the  model  in  better 
context  of  the  network  application. 

a.  Queue  disciplines  and  the  conservation  law 

The  selection  of  the  appropriate  priority  mechanism  is  a  sub¬ 
set  of  the  more  general  problem  of  selecting  a  queue  discipline  from 
the  multitude  of  available  techniques  including  FCFS  (first  cane,  first 
served),  DCFS  (last  cane,  first  served),  selection  at  randan,  etc. 
Kleinrock  (KL64)  has  shown  that  the  average  queueing  delay  is  independ¬ 
ent  of  the  queue  discipline  under  the  following  "work"  conserving 
conditions : 

1.  all  messages  remain  in  the  system  until  serviced 

2.  there  is  a  single  server 
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3.  nan-preemptive  (or  preemptive-resume  with  exponential 
service) 

4.  Poisson  arrivals. 


This  result  is  his  well  known  conservation  law  which  states 

that: 


P 


£ 

P-1 


p  W  = constant  with  respect  to  the  queue  discipline  (5.1) 
P  P 


where  P  is  the  number  of  priority  classes,  pp  is  the  traffic  in¬ 
tensity,  and  Wp  is  the  average  queueing  delay  for  units  in  the  pti 
priority  class.*  The  constant  is  evaluated  by  noting  that  for  P 
equal  to  one,  the  system  reverts  to  a  FCFS  discipline.  An  intuitive 
feeling  for  the  meaning  of  the  conservation  law  can  be  gained  by  re¬ 
writing  the  sun  as: 


Ep w  =  y  X  W 

P  P  *-*  P  P  P 
P-i  P-1 


which  by  utilizing  Little's  result  (LI61) : 

E[n]  =  n  =  XW 

(where  n  is  the  expected  number  of  messages  in  the  system) ,  becomes: 


P 


E 

p=i 


p  W  =  /  .  n  x 

p  p  pipp 


(5.3) 


*We  shall  utilize  the  convention  that  the  index  1  has  the  highest 
priority,  with  lower  priority  messages  having  larger  subscripts. 
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This  latter  form  of  the  equation  is  in  terms  of  the  average  value  of 
the  unfinished  work,  and  while  not  a  formal  proof  of  the  conservation 
law,  does  give  a  feeling  for  what  is  being  conserved.  In  essence,  the 
conservation  law  reflects  the  fact  that  the  amount  of  work  remaining  to 
be  done  will  be  independent  of  the  order  of  service. 

In  the  special  case  where  the  service  density  is  the  same  for 
all  classes,  then: 


and  Equation  (5.3)  reduces  to: 

P 

S  XD  wn  =  *  W(FCFS)  =  E[no.  in  queue]  (5.4) 

p=l  p  p 

which  states  that  for  this  case,  both  the  expected  number  of  messages 
in  the  queue  and  the  expected  queueing  delay  are  independent  of  the 
queue  discipline. 

Although  these  queue  disciplines  have  the  same  mean  queueing 
delay,  they  can  be  shown  to  have  significantly  different  delay  variances 
and  therefore  produce  differing  operational  results.*  However,  for  the 
special  case  in  which  one  knows  the  service  requirements  in  advance,  a 
significant  reduction  can  also  be  obtained  in  the  mean  queueing  delay, 
and  fortunately  the  message  switching  application  is  one  such  case.  We 
shall  exploit  this  situation  to  the  fullest  possible  extent  in  our 
analysis,  in  considering  both  the  message  delay  and  buffer  storage 
aspects  f  the  network.  Note  that  this  reduction  in  the  average 

it 

Vhitney  tabulates  these  differences  for  a  nurrber  of  queue  disciplines 
in  Reference  WH70. 
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queueing  delay  and  the  number  in  the  system  is  not  in  conflict  with  the 
conservation  law,  since  we  are  not  changing  the  amount  of  unfinished 
work,  but  rather  are  taking  advantage  of  the  additional  information 
about  the  service  requirements. 

b.  Preemptive  service  disciplines 

Just  as  there  are  several  variations  of  the  queue  discipline 
in  selecting  the  next  message  for  service,  there  are  also  a  number  of 
possible  service  disciplines.  We  are  concerned  here  with  the  preemptive 
aspects  of  the  service  discipline,  i.e. ,  the  situation  when  the  service 
of  a  message  is  prematurely  terminated  to  handle  a  higher  priority  mes¬ 
sage.  The  possible  preemptive  situations  can  be  categorized  into  four 
c^ifferent  types  based  on  the  decision  algorithm  to  preempt  and  on  the 
recovery  mechanism  for  the  preempted  message.  The  former  are: 

1.  preempt  immediately  if  a  lower  priority  class  message  is 
being  serviced 

2.  decide  to  preempt  or  not  based  on  seme  knowledge  of  the 
remaining  or  expended  service  on  the  message  being 
handled. 

and  the  recovery  mechamisms  are: 

a.  a  preempted  service  must  be  restarted  frem  its  original 
starting  point 

b.  a  preempted  message  can  be  restarted  at  the  point  it  was 
interrupted  (or  sane  roll-back  point  other  than  its 
origin.) 
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Although  the  network  being  considered  does  not  utilize  any 
form  of  preemptive  priorities,  both  cases  la  and  2a  would  be  feasible, 
while  the  preemptive- resune  cases  would  not  be  unless  major  changes 
were  made  in  the  IMP- to- IMP  packet  handling  protocol,  buffer  management 
methods,  and  header  information.  However,  che  segmentation  of  messages 
into  packets,  as  will  be  discussed  in  Section  5.2,  provides  many  of  the 
preemptive  priority  advantages,  so  that  the  network  application  of  pre¬ 
emptive  priorities  was  not  considered  to  be  a  fruitful  area  for  further 
investigation. 

c.  The  round  trip  delay  considerations  of  conversational 
messages 

A  primary  concern  of  the  analysis  will  be  with  the  added 
delay  which  the  network  introduces  to  the  interactive  conversational 
carrnuni cations  between  a  user  and  the  computer.  Therefore,  we  must 
consider  the  delays  added  to  the  total  round  trip  of  the  user-to-ccm- 
put-er  cannand  aivd  the  computer- to- user  response.  Since  experience  in¬ 
dicates  that  these  two  types  of  messages  differ  in  length  by  an  order 
of  magnitude,*  and  because  delays  in  either  direction  of  transmission 
are  of  equal  significance  to  the  terminal  user,  we  must  consider  both 
sources  of  delay  in  selecting  a  priority  discipline  to  minimize  the 
average  interactive  delay. 

The  total  delay  that  the  network  adds  to  the  interactive 
response  will  also  include  other  factors  such  as  propagation  and  modem 
delays,  but  these  effects  are  outside  of  our  domain  of  control,  so  we 

*Test  data  taken  by  Bell  Telephone  Laboratories  indicates  a  ratio  of 
about  12  to  1  for  such  traffic.  (See  References  FU70  and  JA69. ) 
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queueing  delay  and  the  number  in  the  system  is  not  in  conflict  with  the 
conservation  law,  since  we  are  not  changing  the  amount  of  unfinished 
work,  but  rather  are  taking  advantage  of  the  additional  information 
about  the  service  requirements. 

b.  Preemptive  service  disciplines 

Just  as  there  are  several  variations  of  the  queue  discipline 
in  selecting  the  next  message  for  service,  there  are  also  a  mmber  of 
possible  service  disciplines.  We  are  concerned  here  with  the  preenptive 
aspects  of  the  service  discipline,  i.e.,  the  situation  when  the  service 
of  a  message  is  prematurely  terminated  to  hendle  a  higher  priority  mes¬ 
sage.  The  possible  preenptive  situations  can  be  categorized  into  four 
different  types  based  on  the  decision  algorithm  to  preempt  and  on  the 
recovery  mechanism  for  the  preempted  message.  The  former  are: 

1.  preempt  immediately  if  a  lower  priority  class  message  is 
being  serviced 

2.  decide  to  preempt  or  not  based  on  seme  knowledge  of  the 
remaining  or  expended  service  on  the  message  being 
handled. 

and  the  recovery  mechamisms  cure: 

a.  a  preempted  service  must  be  restarted  frem  its  original 
starting  point 

b.  a  preenpted  message  can  be  restarted  at  the  point  it  was 
interrupted  (or  some  roll-back  point  other  than  its 
origin. ) 
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shall  concentrate  on  those  delays  which  can  be  reduced  by  careful 
selection  of  parameters  and  message  handling  techniques.  The  primary 
source  of  delay  which  one  can  manipulate  is  that  due  to  queueing,  and 
henoe  this  leads  to  our  emphasis  on  that  aspect  of  the  problem.  Our 
model  will  be  designed  to  address  this  problem  and  will  include  the 
above  nentioned  variation  between  the  command  and  response  message 
lengths. 

Hie  service  requirement  of  a  message  is  a  function  of  its 
length  and  the  channel  capacity,  such  that: 

x  =  i/C 

where: 

x  =  service  time  requir  1  for  serial  transmission 

i  =  known  nessage  length  in  bits 

C  =  channel  capacity  in  bits  per  second 

We  consider  here  only  die  actual  text  portion  of  messages ,  i . e . ,  the 
overhead  due  to  oartnuni cations  control  characters ,  header  information, 
ard  the  check  sum  is  ignored  sinoe  they  are  the  same  for  all  messages 
and  are  therefore  merely  an  additive  constant.  (Hiis  assumes  contigu¬ 
ous  messages,  and  will  change  somewhat  for  the  segmented  message  case 
considered  in  Section  5.2.)  User-to-ccmpufcer  message  lengths  will  be 
assumed  to  be  randan  variables,  with  service  requirements  drawn  fran 
tne  density  function;  by(x) ,  and  similarly  the  ccmputer-to-user  re¬ 
sponse  service  times  are  assumed  to  be  frem  the  density,  bc(x). 

Wb  will  assure  that  there  are  an  equal  nunber  of  user  oermands 
and  computer  responses,  although  we  allow  for  an  arbitrary  difference 
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in  their  distributions  and  mean  values.  This  assumption  does  not  take 
into  account  computer  output  that  occurs  with  several  bursts  in  the 
total  response;  a  situation  which  is  analogous  to  the  segmented  message 
case  of  Section  5.2.  However,  there  is  a  compensating  user  effect, 
namely  the  character-by-character  transmission  of  seme  terrinals.  In 
this  mode  of  operation  each  character  is  sent  as  a  separate  user-to- 
oernputer  message,  with  a  response  being  generated  only  after  the  entire 
multi-character  command  has  been  entered.  Therefore,  the  assumption  of 
an  equal  number  of  messages  from  the  user  and  the  computer  seems  to  be 
reasonable,  and  it  allows  us  to  simplify  the  analysis  by  considering 
the  oernposite  density  function: 

b(x)  =  0.5b,  (x)  +  0.5b  (x)  (5.6) 

as  the  effective  service  time  density  of  the  interference  traffic  that 
a  "tagged"  message  (i.e.,  one  which  we  will  follow  through  the  system) 
will  encounter  upon  arrival  at  the  service  facility.  If  we  consider  a 
two-priority  system,  i.e.,  the  case  of  P  equal  to  two,  then  the 
probability  that  a  "tagged"  user-to- computer  command  is  of  the  priority 
class  will  be  denoted  as  ay, 

au  =  Pr[a  user-to-oomp.  mes.  is  of  the  priority  class)  (5.7a) 

and  similarly,  any  computer- to-user  response  will  be  of  the  same  class 
with  probability,  ac,  where: 

a,  =  Pr [a  ccmp. -to-user  mes.  is  of  the  priority  class)  (5.7b) 

The  selection  process  for  determining  membership  in  the  priority  class 
can  be  based  on  message  length,  urgency,  or  ary  other  criteria,  and 
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will  result  in  sene  average  queueing  delay  for  priority  messages  which 
will  be  denoted  by: 

=  E [queue  delay  |  priority  message]  (5.8a) 

with  the  average  queueing  delay  for  non-priority  messages  being  denoted 
by: 

=  E [queue  delay  |  non-priority  message]  (5.8b) 

At  any  given  node,  the  expected  queueing  delay  for  a  user-to-canputer 
message  will  be: 

E[Wu  |  user-to-canputer  mes. ]  =  W1  +  (1  -  au)W2  (5.9a) 
and  for  the  computer  response,  the  expected  queueing  delay  will  be: 

E [W  |  computer- to- user  mes.]  =  a  W.  +  (1  -  a  )W_  (5.9b) 

C  C  X  C  4 

The  queueing  delay  of  interest  is  the  total  for  the  user  cormand  and 
the  oarputer  response,  and  at  a  given  node  along  the  store-and- forward 
path,  is: 

E[Wu  +  Wc]  =  (au  +  ac)Wx  +  [2  -  (o^  +  ac)]W2  (5.10) 

If  we  define  the  parameter,  a,  as  the  probability  that  an  arbitrary 
message  is  of  the  priority  class,  then  for  equal  rates  for  user  and 
oemputer  generated  messages, 

a  =  0.5a  +  0.5a 
u  c 

or: 

au  +  ac  =  2a 

Substituting  into  Equation  (5.10),  we  obtain: 
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E[wu  +  wc]  =  2  [a  +  (1  -  ot)W2] 


(5.11) 


which  shows  that  the  expected  contribution  to  the  total  comand-response 
delay  is  merely  twice  the  expected  unconai  tional  queueing  delay  at  the 
rode,  and  therefore  allows  one  to  utilize  conventional  queueing  theory 
results  in  the  overall  delay  analysis. 

The  introduction  of  a  composite  service  time  density  means 
that,  at  best,  the  queueing  system  nodel  can  be  of  the  M/G/l  type  (i.e., 
Markovian  arrivals,  general  service  times,  and  a  single  server).  There¬ 
fore,  such  a  model  does  not  have  the  memory  less  properties  of  the  M/M/1 
system  which  make  queueing  network  analysis  tractable  as  discussed 
earlier  in  Section  1.4,  and  the  use  of  the  model  to  develop  procedures 
for  delay  minimization  should  be  considered  to  be  a  local  optimization 
which  should  lead  to  good,  but  not  necessarily  optimal,  overall  network 
performance. 

5.1.1  The  General  Analysis  of  a  Two  Priority  Class  Queueing  System 

The  effect  of  priorities  can  he  to  either  give  better  service 
to  certain  selected  messages,  or  to  reduce  the  average  delay  for  mes¬ 
sages  in  general.  Fortunately,  if  the  higher  priority  class  of  messages 
is  also  of  shorter  length,  then  one  obtains  both  improvements  simultane¬ 
ously.  However,  longer  messages  will  have  larger  average  delays  in  this 
priority  situation  as  compared  to  a  non-priority  system.  We  shall  con¬ 
sider  the  effect  on  each  class  of  message  in  Section  5.1.2,  bul  for 
now,  we  are  concerned  only  with  the  overall  average  message  queueing 
delay. 
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In  the  following  theorem  we  shall  investigate  the  change  in 
this  overall  expected  queueing  delay  for  an  arbitrary  division  of  mes¬ 
sages  into  two  priority  classes.  The  result  will  include  the  case  of 
shorter  messages  having  higher  priority,  but  will  also  be  valid  for 
other  cases  such  as  giving  priority  to  long  messages  or  a  random  assign¬ 
ment  of  priorities. 

Theorem  5.1.1 


For  a  single  server  queueing  system  with  Poisson  arrivals, 
infinite  queueing  space,  and  a  general  service  process,  the 
expected  queueing  delay  (averaged  over  all  messages)  for  a 
two  class  priority  system  will  be: 


W  »  W(FCFS) 


1  -  a  X  x 
1  -  0  x  *i. 


using  any  arbitrary  discrimination  process  to  divide  the  mes¬ 
sages  into  the  two  classes,  where  the  rotation  is: 

Effraction  of  priority  messages]  =  a  (5.13a) 

E [service  time  of  a  priority  message]  =  x^  (5.13b) 

and  in  which: 


X  =  average  arrival  rate 
x  =  E  [service  time  of  an  arbitrary  message ] 

T 

x  =  E  [square  of  the  service  time  of  an  abritrary  messaqe] 
W(PCFS)  =  E [queueing  delay  |  FCFS  discipline] 
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with: 


WfFCFS)  =  X  x*  /  2(1  -  p) 

and: 

p  =  X  x  <  1 

Proof: 

From  Cobham's  result  (COS  4)  we  know  that  -he  expected  delay  for  a 
priority  message  is: 

W1  =  WC  +  "l  *i  (5.14) 

where : 

-  E  [queueing  delay  of  a  priority  message] 

WQ  =  Eftime  to  ccnplete  an  existing  service] 

nx  =  E[nunber  of  priority  messages  in  the  queue] 

and  one  can  sh<"w  that  (e.g. ,  see  Reference  SA61) : 

~2 

WQ  =  X  x  /  2  (5.15) 

Next  we  utilize  Little's  result  (LI61) ,  which  states  that  the  exnected 
number  in  the  system  (or  queue)  equals  the  average  arrival  rate  times 
the  expected  time  in  the  system  (or  queue) .  Applying  these  results, 
with  an  average  arrival  rate  of  priority  messages  of  aX,  we  obtain: 

.  T 

wi  =  +  wi 

or  in  reduced  form: 
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(5. 16) 


W  -  x2/2) 

^  1  -  aXx^ 

By  a  similar,  but  somewhat  more  involved  procedure,  we  can  calculate  the 
expected  delay  for  a  non-pricrity  message.  One  can  show  that: 

w2  =  W0  +  n^  +  nax1  +  n2x2  (5.17) 

which  states  than  an  arriving  non-priority  message  will  expect  to  wait 
for  the  completion  of  the  service  being  performed,  plus  the  service 
of  riie  n,  priority  messages  already  in  queue,  plus  that  for  the  n 
subsequent  priority  arrivals,  and  finally,  that  for  the  n2  non-priority 
messages  that  were  already  in  the  queue  (which  will  each  require  an 
average  of  x2  seconds  of  service).  Using  Little's  result  the  equa¬ 
tion  becomes: 

,  “7 

W2  =  +  aX  W^x^  +  aX  +  (1  -  a)X  W 2x2 


which  upon  combining  terms  becomes: 


w2  = 


(X  ?/2)  +  aX  x.W. 

X  x 

1  -  p 


(5.18) 


where: 

p  -  X  x 

and  with  further  reduction  yields: 


(5.19) 


The  expected  delay  for  an  arbitrary  message  is  then  found  by  combining 
these  two  components,  since  the  unconditional  delay  is: 
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E [delay]  =  W  =  £  E[delay|p  priority  class]  •  Pr[merrfc>er  of  p  class] 
P 

whj.ch  beccnos: 


W  =  a  W  +  (1  -  a)  t-~ 
l  l  -  p 


W  =  <**  .[  1  -  aP 


1  -  aXx, 


(5.20) 


Since  the  first  factor  is  the  PCFS  queue  delay,  and  replacing  p  by 
Ax  to  give  a  similar  form  to  both  the  numerator  and  denominator,  we 
have: 


W  =  W (PCFS)  • 


1  ~  c\\x 
1  -  oXx. 


(5.21) 


which  completes  the  proof. 

This  theorem  indicates  that  the  expected  queueing  delay  with 
priorities  can  be  either  less  than,  equal  to,  or  greater  than  the  FCFS 
delay  depending  on  the  relative  values  of  and  x.  However,  the 
delay  will  remain  finite  for  all  values  of  p  less  than  unity  since: 


1  -  ctAx 
1  -  ctAx, 


(5.22) 


due  to  the  fact  that: 


x  =  otx^  +  (1  -  a)  x2 


(5.23) 
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Similarly,  the  zero  of  the  function  does  not  occur  in  the  range  of 
p  <  1  since, 

aXx  =  *p  (5.24) 

and  a  is  at  most  equal  to  unity. 

The  result  of  Theorem  5.1.1  was  developed  independently,  but 
was  later  found  to  be  a  generalization  of  an  earlier  result  obtained  by 
Morse  (M058)  and  another  described  by  Conway,  et  al.  (0067).  In  each 
of  these  cases,  the  authors  were  considering  a  particular  scheme  of 
priority  classification,  and  neither  recognized  that  the  result  is 
applicable  to  any  arbitrary  separation  of  the  items  requiring  service 
into  two  classes,  with  the  only  parameters  of  concern  being  the  fraction 
of  the  items  that  are  in  the  priority  class,  and  their  average  service 
requirement. 

The  problem  considered  by  Morse  was  the  following.  Priority 
messages  have  an  exponential  service  time  density  with  a  mean  service 
rate  of  y.  Non-priority  service  requirements  are  also  exponential, 
but  with  a  service  rate  By.  Note  that  a  priority  message  may  require 
a  longer  service  time  than  some  non-priority  ires  sage ;  indeed,  the 
priority  messages  may  be  longer  on  the  average  since  B  can  either  be 
less  than  or  greater  than  unity.  Examples  of  such  a  system  would  be 
those  in  which  the  sender  of  the  message  has  paid  a  premium  for  faster 
service,  has  a  degree  of  urgency,  or  outranks  other  users.  Morse  de¬ 
veloped  his  solution  by  considering  the  state  equations  for  the  queues 
and  solving  for  the  expected  value  of  the  queueing  delays  from  the 
generating  functions.  He  then  considered  the  ratio  of  delays  with  and 
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without  the  priorities  and  obtained  as  a  result, 

ratio  =  1  ~  apfo  +  (1  -  a)  (1/6) ]  =  1  ~  a  peff 
1  -  ap  1  -  ap 


(5.25) 


in  which  his  pQff  corresponds  to  our  p,  and  his  value  of  p  is  \/u 
or  our  Ax^.  His  result,  which  was  based  on  exponential  service  times 
for  each  of  two  priority  classes,  is  an  interesting  special  case  of 
Theorem  5.1.1. 

The  problem  described  by  Conway,  et  al.  is  the  same  as  we 
will  consider  in  Section  5.1.2,  i.e.,  the  situation  when  the  two 
priority  classes  are  distinguished,  by  a  threshold  service  requirement, 
such  that  messages  requiring  less  service  than  this  threshold  value  are 
considered  to  be  of  higher  priority.  We  will  discuss  their  result  in 
more  detail  later,  since  the  priority  threshold  case  is  the  problem  of 
particular  concern  to  the  ARPA  network. 


5.1.2 


The  TWo  Priority  Class  Case  With  Priority  Based  on  Message 
Length' 


Ihe  two  priority  class  case  utilizes  a  priority  threshold, 

to  categorize  messages  into  two  groups?  (1)  priority  iressages  for 

which  0  <  x  <  X  ,  and  (2)  non-priority  messages  with  X  <  x  .  For 
i  T 

this  special  case,  the  value  of  a  will  be  a  function  of  X  , 

T 

A 

=  Pr [priority  messagej  =  J  b(x)dx 


=  a 


and: 
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»2  ■  Pr  [non-priority  message]  =  J* b(x)dx 


a2  =  (1  -  a) 

The  general  two  class  priority  result  of  Theorem  5.1.1  applies  here, 
and  becomes  a  function  of  the  priority  threshold,  )C, 


woy  =  W(PCFS)  • 


1  -  a 


(ty 


Xx 


1  -  a 


oyxi^oy 


(5.26) 


where  the  functional  relationships  of  a  and  have  been  shown 

explicitly.  However,  we  shall  drop  this  functional  notation  in  the 

sequel  for  rotational  convenience.* 

We  can  determine  a  number  of  characteristics  of  the  WO^) 

function  from  the  form  of  Equation  (5.26).  For  example,  for  -  0, 

* 

tnere  will  be  no  priority  messages,  i.e.,  a  =  0,  and  we  are  left  with: 


WO^  =  0)  =  W(FCFS) 


(5.27) 


Our  result  was  obtained  independently  from  that  of  Conway,  et  al. 
(0067),  who  also  considered  this  special  case  of  Theorem  5.1.2.  Their 
result  was  of  the  form: 

_  XE[p2]  .  Il  -  pG(d)\ 

Elwl  -  2(1-  p7  (  r-"p1  / 

where:  * 

=  XG  (d)Efp  |p  <  d] 

and  G(pN  is  the  cumulative  service  distribution,  such  that  G(d)  is 
equivalent  to  our  a.  Their  p  and  d  correspond  to  our  x  and  X, 
respectively.  They  noted  that  for  any  G(p),  d  is  an  increasing  1 
function  of  X ,  and  is  always  greater  than  E  [p] .  They  calculated  the 
optimal  priority  threshold  for  the  exponential  and  uniform  densities 
using  the  approach  of  Cox  (0061) . 
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* 


The  value  of  c  can  not  be  greater  than  x  since: 

-  >  /“ 

ocx^  =  J  xb(x)dx  <  J  xb(x)dx  =  x 

0  0 

In  the  limit  as  becomes  infinitely  large ,  ccx^  approaches  the 
value  of  x,  so  that: 

WO^  -*“>)  =  W(PCFS)  (5.28) 

The  general  shape  of  a  plot  of  the  expected  delay,  W,  as  a 
function  of  the  priority  threshold,  X^,,  is  therefore  known  to  be  equal 
to  the  FCFS  value  at  both  extremes  of  and  to  have  seme  lesser 
value  at  intermediate  points.  This  general  shape  implies  a  minimal 
delay  at  seme  optimal  value  of  X^,,  and  we  s'- all  investigate  the  loca¬ 
tion  of  this  optimum  and  the  degree  of  improvement  that  one  can  obtain. 
This  investigation  shall  consist  of  four  parts;  (1)  the  application  of 
Theorem  5.1.1  to  sane  sample  service  density  functions,  (2)  seme  ad¬ 
ditional  theoretical  results  frem  these  observations,  (3)  the  introduc¬ 
tion  of  a  delay  cost  weighting  factor,  and  (4)  seme  experimental  re¬ 
sults  pertaining  to  the  two  priority  class  model. 

5. 1.2.1  Exanples  of  the  Two  Priority  Class  Case 

We  will  consider  several  example  service  time  density  func¬ 
tions  and  the  corresponding  relationships  between  the  queueing  delay, 

W,  and  the  priority  threshold,  X^,.  These  exanples  provide  a  more 
intuitive  feel  for  the  mechanisms  involved  and  for  the  results  that  are 
obtained.  Fortunately,  they  also  provide  seme  new  insights  and  sur¬ 
prising  generalizations  to  the  analytical  results. 
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a.  The  exponential  service  density  case 

If  we  consider  the  special  case  for  which  the  service  time 
density,  b{x),  is  exponentially  distributed, 


b  (x)  =  ye_fJX  0  <  x 

we  cbtain  the  following  function  forns  for  a  and  ax^ 

A  -ux  "A 

a  =  /  ye  4  dx  =  1  -  e 

0 


(5.29) 


15.30) 


and: 


xye”UXdx  =  jj[l  - 


-VOL, 

(1  +  y*p)e  ] 


(5.31) 


If  we  substitute  these  values  into  the  expressions  for  Wp  and  W2  of 
Theorem  5.1.1,  we  can  obtain  the  expected  delays  for  the  priority  and 
non-priority  messages  respectively,  as  a  function  of  \p»  namely: 

w  .  (X?/2) 

1  1  -  aXXp 


or: 


w 


1 


p/y _ 

-yXm 

l  -  p[i  -  (l  +  y^pJe  ] 


(5.32) 


and: 


W 


2 


(5.33) 


These  equations  are  plotted  as  a  function  of  Hp  in  Figure  5.1.1, 
with  p  fixed  at  0.8.  Both  the  delay  and  ^  scales  are  normalized 
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uw  tHJfWI  DJlAV 


Figure  5.1.1  Plots  of  the  Queueing  Delays  for  Priority  and  Non-priority 
Messages  as  a  Function  of  tha  Priority  Threshold,  XT 
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with  respect  to  the  average  service  time. 

The  unconditional  average  delay,  W,  is  alio  shown  in  the 
figure,  where: 

w  =  oWx  +  (1  -  a)W2  (5.34) 

since  a  is  the  probability  that  a  message  is  of  the  priority  class. 
The  variable  a  is  a  function  of  being  equal  to  the  cumulative 
distribution  function, 


a  =  BOy  s  Pr[x  <  jy  (5.35) 

The  optimal  value  of  can  be  seen  to  be  at  about  =1.6,  which 
puts  the  priority  threshold  at  1.6  times  the  average  message  length. 
This  value  of  the  threshold  is  much  higher  than  one  would  intuitively 
expect,  and  as  can  be  seen  frcrn  the  corresponding  value  of  a,  means 
that  about  80%  of  the  messages  in  the  system  would  be  included  in  the 
priority  class  1 

The  optimal  value  of  Hp  is  that  which  minimizes  the 
function,  Wfxy  ,  and  Cox  (C061)  has  shown  that,  although  an  explicit 
form  can  not  be  obtained,  the  optimal  value  of  is  related  to  p 
by  the  equation: 


>*ro  >  1 


(5.36) 


b.  A  comparison  of  results  for  the  hyperexnonential, 
exponential,  and  Erlang  service  densities 

A  broad  range  of  service  time  densities  can  be  modeled  by  the 
use  of  hyperexponential,  exponential,  and  Erlang  functions,  so  these 
densities  were  selected  as  further  examples  of  the  optimal  priority 
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threshold  analysis.  The  hyperexponential  density  was  assumed  to  have 
two  equally  liked  exponential  components  with  service  rates  of  u  and 
yv; 


b(x;y ,M)  =  0.5vie  ^  +  0.5YUe-Yyx  0  <  x  (5.37) 

For  this  service  time  density  function,  the  result  of  'theorem  5.1.1  can 
be  shown  to  be: 


where: 


WO^)  =  W  (FCFS) 


(5.38) 


and: 


W(PCFS)  =  y-5— 
1  -  p 


(1  +  Y2) 

WY(1  +  y) 


t  =  [1  -  (1  + 


vtye  +  i[l  - 


(1  +  YPXJC 


~yuXT, 


(5.39) 

(5.40) 


a  =  BO^,) 


0.5[1  -  e  + 


-YM*p 

0.5 [1  -  e  x] 


(5.41) 


The  corresponding  relationship  between  p  and  tlie  optimal  priority 
threshold,  X__ ,  can  also  be  shown  to  be: 


0.5(e  ^  -t-e 


(5.42) 


which  for  Y  *  1#  becomes  the  exponential  result  of  Equation  (5.36). 

The  form  of  the  hyperexponential  density  of  Equation  (5.37) 
is  useful  when  one  is  concerned  with  the  ratio  of  the  service  rates, 
and  expects  the  mean  service  time  to  vary  as  a  function  of  this  value, 
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Si.ncx»: 


x  =  E[x] 


=  1  +  V 

2yu 


however,  if  a  constant  mean  service  time  is  desired.  Equations 
through  (5.42)  can  be  modified  by  the  substitution  of: 


(5.43) 

(5.38) 


y  = 


i  +  Y 
2yx 


(5.44) 


which  follows  directly  from  Equation  (5.43),  with  the  expected  value 
of  x  being  a  constant. 

The  hyperexponential  density  of  Equation  (5.37)  is  actually  a 
family  of  functions,  if  we  let  the  value  of  y  vary  as  a  parameter. 

The  mean  value  was  given  in  Equation  (5.43)  and  with  the  second  manent 
being: 


E[x2] 


(5.45) 


the  variance  can  be  shown  to  be: 


2  1  2 
«  »  - -  (3  -  2y  +  3y  J 

4  (YU) 


(5.46) 


such  that  the  coefficient  of  variation  is: 


C 


3  -  2y  +  3y' 


1  +  2y  +  y' 


1/2 


(5.47) 


The  coefficient  of  variation  will  reach  its  maximum  value  of  the  square 
root  of  three  at  both  extreme  values  of  y;  namely  at  zero  and  as  it 
becomes  arbitrarily  large,  and  will  be  minimum  of  one  at  y  equal  to 
unity. 
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The  Erlang  density  can  also  bo  considered  to  b^  a  family  of 
functions,  with  the  parameter,  K,  being  any  positive  integer  greater 
than  zero,  such  that: 

b(x;K,u)  =  7k~~i7T  (Kux)K~1e~KlJX  0<x  (5.48) 

For  K  equal  to  unity ,  the  function  reverts  to  the  simple  exponential 
case,  just  as  the  hyperexponential  did  for  y  equal  to  unity.  Thus 
Erlang  densities  represent  one  half  of  the  spectrum  of  functions  which 
includes  the  exponential  as  a  central  function  and  the  hyperexponential 
the  other  half,  with  coefficients  of  variation  of  less  than  unity 
for  the  Erlang  case,  and  greater  than  unity  for  the  hyperexponential. 

We  will  only  consider  the  case  of  K  equal  to  two  for  the 
Erlang  family,  since  a  general  solution  would  require  a  closed  form 
representation  of  the  integral, 

*T 

a  =  7K  1  /  (KUx)K-1e‘KyXdx  (5.49) 

0 

which  is  a  variation  of  the  inccnplete  gantna  function  and  is  known  not 
to  be  expressible  in  closed  form.  However,  the  lack  of  a  general  solu¬ 
tion  is  of  little  concern  because;  (1)  our  a  priori  expectation  is  that 
the  service  densities  will  be  more  hyperexponential- like  due  to  the 
composite  user  and  computer  generated  traffic,  and  (2)  the  effect  of 
priority  handling  is  most  dramatic  when  there  is  a  large  coefficient  of 
variation  for  the  service  times. 

For  the  case  of  K  equal  to  two,  the  queueing  delay  expres¬ 
sion  from  Theoran  5.1.1  becomes: 
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(3.50) 


«<V  -  w>rcre>  •  [{ 


where : 


W  (PCFS) 


(3/4) p 
uTT  -  p) 


$  = 


(2UXJ2 

1  +  2^  +  - 


(5.51) 


l5.52) 


and 


c  =  1 


(1  -  2u*p) 


(5.53) 


This  result  is  plotted  in  Figure  5.1.2  for  the  case  of  p  equal  to 
0.6,  along  with  the  corresponding  delay  functions  for  the  exponential 
and  hypercxpcnentia  1  service  densities.  The  locus  of  the  optimal 
threshold  points  is  also  shown  and  both  sets  of  curves  indicate  the 
spectrum  of  results  which  are  obtained  from  the  Erlanq  and  hyperex¬ 
ponential  families.* 

An  interesting  relationship  can  be  noted  between  the  degree 
of  improvement  relative  to  t»e  FCFS  delay  and  the  location  of  the 
optimal  threshold  relative  to  the  mean.  For  exairple,  for  p  equal  to 
0.6,  the  exponential  density  has  an  optimal  value  of  of  about  1.4 
times  the  mean  and  the  FCFS  delay  is  about  1.4  times  the  delay  at  the 
optimal  location.  This  observed  relationship  can  be  expressed  as: 


The  limiting  case  of  y  equal  to  zero  does  not  behave  as  might  be 
expected  near  the  origin  due  to  the  sharp  cusp  which  forms,  and 
eventually  degenerates  into  an  impulse  function. 
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Xjli 

Figure  5.1.2  The  Locus  of  Optimal  Value  Points  for  Various  Service  Time  Density  Functions 
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w(pcfs)  *rn 


and  will  later  be  shewn  to  be  true  in  general  in  Theorem  5.1.2. 

The  relationship  between  the  value  of  p  and  the  optimal 
threshold  location  can  be  shown  to  be: 


(1  +  U^)e  2U*ro 


(5.54) 


for  the  Erlang  (2)  service  density,  and  this  function  is  plotted  in 
Figure  5.1.3,  wiiich  also  shows  a  conparison  w_th  the  hyperexponential 
family.  The  set  of  curves  includes  the  exponential  result  of  Cox 
(0061) ,  which  again  displays  an  intermediate  behavior  between  that  of 
the  hyperexponsntial  and  Erlang  functions. 

c.  A  conparison  of  three  p.d.f.'swith  same  moan  and  variance 

The  exponential,  hyperexponential ,  and  Erlang  functions  were 
of  interest  since  they  combine  to  cover  a  range  of  possible  distribution 
shapes.  However,  they  also  har/e  varying  moments  and  it  is  difficult  to 
separate  the  effects  that  are  dependent  on  density  shape  versus  those 
that  are  a  function  of  the  moments.  To  obtain  some  measure  of  this 
difference  in  effects,  we  shall  consider  three  density  functions  that 
have  the  same  first  and  seoond  moments,  but  that  differ  significantly 
in  their  shape. 

We  know  from  the  Follaczek-ttunchine  formula  that  the  PCFS 
queue  delay  for  all  three  densities  will  be  the  same,  since  this  delay 
is  a  function  of  the  first  two  moments  of  the  service  time  for  an  M/G/l 
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Figure  5.1.3  The  Relationship  Between  the  Values  of  p  and  the  Normalized 
Priority  Tnreshold  for  a  Spectrum  of  Density  Functions 
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system,  i.e.: 


E  (queue  delay]  = 


~1 

x 

2x 


(5.55) 


Therefore ,  the  W  versus  X  curves  for  all  three  of  the  densities 

T 

will  have  the  same  starting  and  asymptotic  values  since  this  value  is 
the  PCFS  delay.  Our  real  concern  will  therefore  be  with  the  shape  of 
the  W  versus  XT  curve  near  the  optimal  value  and  with  the  resulting 
latitude  in  the  selection  of  the  priority  threshold. 

Hie  three  functions  which  we  will  consider  are  the  hyper¬ 
exponential  density,  a  truncated  1/x  density,  and  a  discrete  distribu¬ 
tion,  with  each  distribution  having  a  mean  value  of  30,  a  variance  of 
1350,  and  a  resulting  coefficient  of  variation  of  1.22.  The  three 
functions  are: 


1.  b(x)  =  0.5ye-yx  +  0.5YUe_YUX  ,  0  <  x  (5.56a) 

for  \i  =  1/15  and  y  =  1/3 

2.  b(x)  «  l/(x  In  M),  l<x<M  (5.56b) 

for  M  =*  150 

3.  b(x)  =  0.56 (x  -  5)  +  0.256 (x  -  18)  +  0.256 (x  -  98)  (5.56c) 

Where  the  delta  function  6  (x  -  a)  is  non-zero  only  for  x  =  a,  where 
it  has  unit  probability  area.  The  three  functions  are  plotted  in 
Figure  5.1.4  and  can  be  seen  to  differ  in  shape  by  a  considerable 
degree.  Figure  5.1.5  shews  the  resulting  plots  of  average  queueing 
delay  versus  the  location  of  the  priority  threshold,  and  these  three 
curves  are  surprisingly  similar,  but  do  not  have  the  exact  same  delay 
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Figure  5.1.4  The  Three  Density  Functions 

reduction.  We  observed  in  the  examples  of  Dart  (b) ,  that  the  delay 
reduction  and  the  optimal  priority  threshold  were  related  by: 

W(FCFS)  _  '\o 

X 

Wa  -nail  later  generalize  this  result  (in  Theorem  5.1.*),  and  it  will 
therefore  shew  that  the  optimal  threshold  differs  only  slightly  for  the 
three  distributions. 
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Figur*  6.1.5  A  Comparison  of  Quouoing  Dotty  Vanui  Priority  ThraahoM  for  tha 
Hyporaxponontiai,  Diioroto,  and  1/x  Sarvieo  Tima  Diatribotiom 


5.1.22  Further  Analysis  of  the  TV*o  Priority  Class  Case 

A  close  look  at  the  priority  analysis  for  the  various 
different  servioe  time  density  functions  s’jggests  several  areas  of 
possible  generalization.  These  observations  include  the  equivalence  of 
the  two  ratios, 

W(FCFS)  _  *ro 
x 

and  the  similarity  of  the  equations  relating  o  and  far  various 

servioe  densities.  We  shall  show  that  one  can  indeed  generalize  these 
relationships  in  the  following  analysis. 

Theorem  5.1.2 

The  ratio  of  the  minimal  queueing  delay,  W()^) ,  to  the  FCFS 
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delay,  is  equal  to  the  inverse  of  the  ratio  of  to  the 
mean  service  time,  x,  for  any  arbitrary  service  density. 


w%> 

W(PCFS) 


(5.57) 


Proof: 


The  optimal  priority  threshold  location,  is  found  by  differentiat¬ 
ing  the  delay  expression  of  Equation  (5.12), 


WO9 


W0(l  -  aXx) 

(1  -  P)  (1  -  aA^) 


However,  one  would  expect  that  this  differentiation  could  only  be 
carried  out  for  a  particular  service  density,  b  (x) .  If  we  proceed  with 
the  differentiation  in  general ,  we  must  evaluate  the  roots  of : 


dW  d 
to?  =  dXp 


WQ(1  -  ap) 

(1  -  p) (1  -  aAx^) 


n 


(5.58) 


The  WQ  and  1  -  p  factors  are  not  functions  of  and  do  not  need 

to  be  considered.  We  are  left  with: 


1  -  gp 
1  ~  aXXj, 


=  0 


Where 


a 


b(x)dx 


0 


and: 


(5.59) 


(5.60) 
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(5.61) 


ax 


1 


0 


Ihe  differentiation  results  in: 

p(i  -  -  Ml  ~  ap)jjg^-(ax^)  =!  o 

or  when  we  substitute  for  a  and  co^,  as  above: 

.  fa  JL, 

P (1  a**l^d)^  f  b(x)dx  -  X(1  -  ap)g^-  f  xb(x)dx  =  0 

Ihe  differentiation  of  the  integrals  is  easily  handled  by  the  relation¬ 
ship  that: 

y 

/  f(z)dz  =  f  (y) 


and  when  ocrmon  terns  are  factored  out,  we  are  left  with: 

P(1  -  aXXj^)  -  XX^(1  -  ap)  =  0  (5.62) 

The  terms  within  the  parentheses  are  those  fran  the  original  expression, 
and  are  therefore  the  subject  of  our  investigation.  In  particular,  we 
are  interested  in  the  ratio: 

-1-  aP  = 

1  -  aXx^ 

Since  we  knew  fran  the  definition  of  p 
p  *  Xx 
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Then: 


1  -  ap  _  x 

1  -  oXxj  (5,63) 

which  when  substituted  into  the  delay  equation  produces: 

w 

w,xro>  =  ’  |3^J  -  w(rcFS»/|^nj 

This  is  then  a  proof  of  the  theorem. 

Corollary  1 

The  optimal  priority  threshold  will  always  be  greater  than  or 
equal  to  the  mean  service  requirement,  regardless  of  the 
service  time  distribution. 

Proof: 

The  theorem  proves  that: 

W(PCFS)  *T0 
x 


As  the  arrival  rate,  A,  becomes  very  small,  we  kna’  that  WO^)  be¬ 
comes  approximately  equal  to  WfPCFS),  since  frcm  Theorem  5.1.1, 


W(PCFS) 


1  -  aAx. 

lim  - - -  =  i 

HI  -  aAx 


(5.64) 


and  that: 


W(PCFS)  >  W(J^0)  V  A 


(5.65) 


because  x^  <  x.  Therefore,  and  x  are  related  by  the 
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inequality, 


(5.66) 


with  equality  for  the  special  cases  of  A  =  0  or  for  the  degenerate 
case  of  x1  =  x  ■  constant.  This  carpletes  the  proof  of  the  corollary. 

Another  relationship  which  seemed  to  have  sere  possible 
generality  was  the  function  relating  p  to  with  an  observed  form 

of: 


i  .  .  ,  f(*ro  -a 
P  *ro-* 

Ihis  relationship  is  generalized  in  the  following  theorem. 
Theorem  5.1.3 


(5.67) 


The  general  relationship  between  the  optimal  priority 
threshold,  and  the  traffic  intensity  is: 


1-T  ■  (1~a)[X2~  V 


(5.68) 


for  ary  arbitrary  service  density,  b(x)  where  a  is  the 
fraction  of  messages  that  are  of  the  priority  class,  and 
is  the  average  service  time  of  a  non-priority  message. 


Proof: 


We  knew  from  Equation  (5.62)  that, 
p[l  -  oAxJ  «  ^.qU  ~  crp] 
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where  a  and  ax^  are  ccnputed  at  from  Fljuations  (5.60)  and 

(5.61). 

Fxpanding  the  above  equation,  we  obtain: 

P  -  oAXjP  +  aX^o  =  XXp0 

or,  by  recalling  that  p  =  Ax, 

A  (x  -  axjp  +  aX^pl  = 

which  results  in: 

[a*T0  '  “V0  =  *T0  "  X 
If  we  solve  for  1/p  we  obtain: 

i  _  a«ro  ~  xl)  (5., 

°  V* 

which  by  adding  and  subtracting  like  quantities,  one  can  write  as: 

1  _  ^0  "  X  +  a(XT0  ~  Xl}  ~  (*T0  ~  X) 

P  *ro  "  x  *ro  "  x 


or: 


1  (x  -  ax^)  -  (1  -  a)>4>n 

P  ^ 


*T0 


-  x 


We  recognize  from  Equation  (5.23),  that: 


x  -  ox,  =  (1  -  a)x. 
x  Z 


(5.70) 


(5.71) 


such  that 
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which  is  the  desired  result. 

Theorem  5.1.3  can  be  utilized  to  determine  the  plot  of  3^ 
versus  p  for  any  given  service  time  density  function.  For  the  ex¬ 
ponential  case  considered  by  Cox  (0061) ,  we  can  write  the  result  by 
inspection  since  we  know  that: 


a<*ro>  =  B(}W  = 


-^o 


(5.72) 


and  the  mean  of  the  exponential  tail  is, 


*2  =  *ro  +  x 

such  that: 


xe~^ 
Xro  -  x 


(5.73) 


This  equation  is  equivalent  to  the  result  given  in  Equation  (5.36)  and 
plotted  in  Figure  5.1.3. 

5. 1.2. 3  Introduction  of  a  Delay  Cost  Function 

A  surprising  aspect  of  the  priority  threshold  analysis  of 
Section  5.1.2  was  the  fact  that  a  large  fraction  of  messages  should  be 
included  in  the  priority  class.  A  more  intuitive  approach  might  select 
only  5%  to  20%  of  the  messages  for  such  preferential  treatment,  and 
while  intuition  should  not  rule  over  analysis,  it  does  signal  a  poten¬ 
tial  oversight  in  the  model. 
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If  one  considers  the  various  classes  of  messages  that  might 
be  serviced  by  a  network,  the  list  would  include  the  short  messages 
such  as  single  character  echoplex  traffic,  terse  ccrmands  (such  as 
/LOGIN) ,  short  lines  of  code,  and  brief  ccmputer  responses,  and  would 
cover  a  wide  range  of  message  sizes  up  to  the  transmission  of  large 
files.  These  classes  are  ordered  by  message  length,  but  this  same 
ordering  applies  reasonably  well  to  one's  response  time  expectations, 
and  therefore  a  weighting  of  delay  as  a  function  of  the  message  length 
seems  appropriate .  One  such  weighting  is  introduced  by  the  utilization 
of  a  delay  cost  function,  c(x),  which  is  defined  as  the  "cost"  of 
delaying  a  message  by  one  second  given  that  the  message  requires  x 
seconds  of  service.  This  cost  can  be  expressed  in  arbitrary  units,  and 
is  merely  a  relative  weighting  of  the  delay  values  for  the  various 
message  lengths. 

An  alternative  approach  would  be  to  implement  N  priority 
classes  for  the  different  message  classifications,  but  the  cost  func¬ 
tion  approach  may  allow  us  oo  accomplish  nearly  the  same  effect  without 
the  additional  overhead  of  a  multiple  priority  scheme.*  We  consider 
only  cost  functions  which  are  a  monotonicallv  non- increasing  function 
of  the  service  requirement,  x,  and  therefore  a  family  of  exponential 
curves  is  felt  to  be  adequate,  but  they  should  be  normalized  to  have  a 
canmon  expected  cost  over  all  message  lengths,  with  a  convenient 

* 

Cox  and  3nith  (C061)  consider  the  case  of  N  priority  classes,  with 
the  cost  of  delay  for  any  message  in  the  ith  class  being  w^  units  of 
cost  per  second.  In  our  example,  the  cost  function  is  continuous, 
rather  than  being  discrete  by  class. 
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normalization  being: 


E [c (x) ]  =1  (5.74) 

This  expected  value  will  be  a  function  of  the  service  time  density, 
b(x) ,  but  for  the  exponential  cost  function,  the  expected  value  takes 
on  a  particularly  interesting  form  as  shewn  in  the  following  theorem. 

Theorem  5.1.4 

An  exponential  weighting  for  the  oost  of  queueing  delay  as  a 
function  of  the  service  requirement,  x,  and  normalized  such 
that  the  expected  cost  is  unity,  is: 

c(x)  =  e-Kx/B*(K)  (5.75) 

where  B*  (K)  is  the  Laplace  transform  of  the  service  time 
density,  and  K  is  an  arbitrary  positive  constant  that  con¬ 
trols  the  degree  of  the  weighting. 

Proof: 

An  exponential  cost  function  is  desired  of  the  form: 

c(x)  *  a  e  ^  ,  0  £  x  (5.76) 

with  the  constraint  that: 

E[c  (x)]  =  1. 

The  expected  value  of  the  cost  function  is,  in  general : 

E[c(x)]  =  |  c(x)b(x)dx  (5.77) 

With  the  exponential  form  of  c(x) ,  we  obtain: 
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E(c(x)]  =  J  a  e  k*b(x)dx  (5.78) 

0 

which  can  be  recognized  as  being  of  the  form  of  a  Laplace  transform, 


L(b(x) }  =  B*(s)  =  /~e“SX b(x)dx  (5.79) 

0 

which  for  s  =  K  results  in: 

E(c(x)  ]  =  a  B*(K) 

Since  this  expected  value  is  equal  to  unity,  we  have  the  relationship: 

a  =  1/B* (K)  (5.80) 

and  therefore  the  generalized  exponential  cost  function  is  found  to  be: 

c(x)  =  e'^/BMK)  (5.81) 

which  proves  the  theorem. 

For  the  exponential  service  density, 

b(x)  =  ye  yX,  0  <  x 

the  Laplace  transform  is: 


B*(s)  = 


s  +  y 


and  the  cost  function  becomes: 


eM  =!^e-Kx 


(5.82) 


A  more  convenient  notational  form  results  if  we  make  the  substitution 
of  variables, 
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k*  =  k/u 


(5.83) 


such  that: 

c(x)  =  Of  +  l)e'k,l“  <5'84) 

A  family  of  exponential  cost  functions  are  thereby  produced,  ranging 
fran  the  extreme  case  of  a  constant,  c(x)  equal  to  unity,  for  k' 
equal  to  zero,  to  very  sharply  peaked  functions  near  the  origin.  The 
latter  vould  heavily  vasight  the  delay  for  messages  requiring  very  short 
service  times,  as  shown  in  Figure  5.1.6. 


X 


Figure  5.1.6  Examples  of  the  Exponential  Family  of  Cost  Functions 


The  introduction  of  a  delay  cost  function  changes  the  priority 
analysis  of  Section  5.1.2,  such  that  for  a  message  requiring  x  seconds 
of  service,  the  expected  delay  cost  is: 
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i 


E [delay  cost  |  xl  =  c(x)  •  E [queue  delay  j  x] 


(5.85) 


For  the  two  priority  class  system, 
E  [queue  delay  |  x  < 

and: 


E[que>’£  delay  I  x  >  V  -  W2 

such  that  the  unconditional  delay  cost  function  becomes: 


E  [delay  cost]  =W .  J  c(x)b(x)dx 

0 

We  shall  introduce  the  notation: 

ac  =  /  c(x)b(x)dx 
0 


+  W 2  J  c(x)b(x)dx 


(5.86) 


resulting  in: 

E  [delay  cost]  =  +  (1  -  ac)W? 

The  equations  for  and  W2  are  not  functions  of  c  (x) ,  and  there¬ 
fore  are  as  in  Equations  (5.16)  and  (5.19).  Substitution  and  simplifi¬ 
cation  produces: 

WQ  1  -  ap 

E [delay  cost]  =  r - -  •  - - —  (5.87) 

1  "  p  1  -  aXx 
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in  which  both  a  and  a  appear.  For  the  exponential  cost  function 

c 

and  the  exponential  service  density,  the  equation  for  ac  beoanes: 

ac  =  j\(,  +  l)e-(k'+1)Uxdx 
0 


or: 


(k'+Uu^ 


(5.88) 


The  resulting  cost-weighted  delay  as  a  function  of  the  priority  thresh¬ 
old,  is  shown  in  Figure  5.1.7.  The  reader  should  recall  that,  with  the 


5 

D 


Figure  5.1.7  Optimal  Priority  Threshold  Curves  for  Several 

Parameter  Values  of  the  Exponential  Cost  Function 
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exception  of  k'  =  0,  the  delay  is  not  scaled  in  normal  time  units,  so 
the  apparent  delay  reduction  for  k'  >  0  is  merely  due  to  the  lesser 
weighting  of  the  delay  for  long  messages.  However,  the  net  effect  of 
the  cost  function  is  to  decrease  the  optimal  priority  threshold,  x^ 
which  also  reduces  the  fraction  of  the  messages  in  the  priority  class. 
This  reduction  is  consistent  with  one's  intuitive  expectations,  which 
implies  that  sane  similar  cost-weighting  may  be  inherent  in  the 
intuition. 

5 . 1 . 2 . 4  Reduction  in  Buffering  Needs  Due  to  Priority  Handling  of 

Messages 

Little's  result,  n  =  XW,  would  lead  one  to  believe  that  we 
can  reduce  the  buffering  needs  at  the  same  time  that  we  reduce  the 
expected  queueing  delay,  since  the  number  of  messages  in  the  queue  is 
proportional  to  the  average  queueing  delay.  However,  the  extent  to 
which  the  delay  reduction  results  in  a  similar  reduction  in  the  buffer¬ 
ing  requirements  is  dependent  on  the  storage  allocation  scheme.  For 
example,  if  storage  is  allocated  with  an  arbitrarily  small  number  of 
bits  in  a  segment,  then  the  priority  system  will  have  no  storage  ad¬ 
vantage  since  bits  will  be  freed  at  the  service  rate  of  the  channel 
regardl'  js  of  whether  long  or  short  messages  are  being  served.  The 
other  extreme  is  when  storage  is  allocated  in  fixed  length  blocks,  in 
which  case  serving  short  messages  frees  these  blocks  of  storage  most 
rapidly.  These  two  allocation  schemes  are  bounds  on  the  effect  of  pri¬ 
orities,  and  will  be  discussed  in  more  detail  in  the  following 
paragraphs . 
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a.  The  continuous  storage  allocation  rodel 


Although  we  know  from  the  above  argument  that  introduction  of 
priorities  should  not  effect  the  storage  considerations  for  this  case, 
let  us  look  at  the  analysis  in  more  detail  since  it  brings  out  several 
interesting  aspects  of  the  problem.  The  expected  storage  requirement 
is: 

E [storage  requirement]  *  Cfn^  +  n^)  (5.89) 

where  C  is  the  service  rate  in  bits  per  second.  Using  Little's  re¬ 
sult  again  we  can  obtain: 

E [storage  requirement]  *  MdWjQCj  +  (1  -  a)W2Cx2)  (5.90) 

in  which  Cx^  is  the  expected  length  (in  bits)  of  a  priority  message , 
and  Wx  is  the  expected  time  that  such  a  injssage  requires  this  storage 
due  to  its  queue  delay.  (Storage  is  also  required  during  service  and 
until  an  ackncwledg  ment  has  been  received  but  we  are  not  considering 
these  effects  here.)*  These  products  of  storage  needed  and  the  time 
period  fur  which  it  is  held  form  a  measure  of  buffer  utilization, 
namely  the  bit-seconds  of  storage  requirement. 

If  we  substitute  for  the  values  W^,  W, ,  and  x2  and  sim¬ 
plify  the  resulting  expression  we  obtain: 

E (storage  requirement]  =  XCx^— (-.91) 


The  time  required  for  serial  transmission,  propagation  delays,  and 
acknowledgement  return  are  independent  of  the  factors  being  considered 
here. 
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which  is  merely  the  ECFS  storage  requirement.  This  result  could  have 
been  obtained  directly  from  the  coaservation  law  since  this  law  is 
based  on  the  fact  that  the  expected  amount  of  unfinished  work  in  the 
queue  is  independent  of  the  queue  discipline. 

b.  The  fixed  length  storage  allocation  model 

If  all  message  lengths  are  less  than  seme  finite  maximal 
length,  L,  and  if  storage  is  allocated  in  segments  of  this  maximal 
length,  then  the  expected  storage  requirement  is: 

Efstorage  requirement]  =  L[n^  +  n2]  (5.92) 

or  from  Little's  result, 

E  [storage  requirement]  =  aXW^L  +  (1  -  a)  XW^ 

w’lich  upon  substitution  for  and  W2  becomes: 

W 

Efstorage  requirement]  =  XL  ---  - - —  (5.93) 

1  '  p  1  -  aXxx 

The  expected  reduction  in  the  storage  requirement  due  to  the  priorities 
is  the  same  as  the  reduction  in  the  expected  queueing  delay,  as  would 
be  expected  since  in  this  case  the  storage  requirement  is  directly 
proportional  to  the  number  of  messages  in  the  system.  In  fact,  Equation 
(5.93)  could  have  been  obtained  directly  from  Equation  (5.12)  and 
Little's  result,  since  the  proportionality  factor  is  know  to  be  L. 

Since  this  is  the  same  reduction  factor  as  was  previously 
seen  in  terms  of  the  average  queueing  delay,  all  of  those  plots  are 
equally  applicable  to  the  buffer  considerations.  However,  one 
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additional  plot  is  shown  in  Figure  5.1.8  to  indicate  the  typical  range 
of  values  of  n,  and  to  show  the  relative  insensitivity  of  the  selec¬ 
tion  of  the  priority  threshold.  Even  when  the  threshold  is  varied  from 
one-half  to  twice  the  optimal  value,  the  improvement  factor  is  only 
slightly  degraded  for  this  exanple  case. 


Figure  5.1.8  The  Effect  of  Priorities  on  the  Buffer  Requirements  for  the  Cere 
of  Fixed  Length  Buffer  Allocation 
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5. 1.2. 5  Experimental  Verification  of  the  Priority  Model 

The  priority  model  was  evaluated  via  an  experiment  in  which 
the  priority  threshold  parameter  of  the  IMP  was  varied  and  the  resulting 
queueing  delays  were  measured  for  several  values  of  the  traffic  inten¬ 
sity,  p.  The  values  of  p  were  selected  using  the  techniques  of 
Section  4.1.3,  with  the  number  of  active  generators  being  selected  as 
the  alterable  parameter.  (Changes  of  the  message  lengths  would  have 
oorrplicated  the  corparison  with  the  theoretical  results  since  both  a 
and  x  would  vary. )  The  set  of  simulated  parameter  values  were  selec¬ 
ted  from  Figure  4.1.3,  with  the  simulated  values  of  p  of  0.3,  0.5, 

0.6,  and  0.8  being  generated  by  the  selection  of  5,  8,  10,  and  14 
messare  generators  respectively.  The  time  parameter,  T  ,  was  set 

a 

accordingly  at  200  msec,  for  all  of  the  tests.  An  average  message 
length  of  320  bits  was  selected  as  a  compromise  between  (1)  desiring  a 
reasonably  large  variable  corponent  for  the  message  service  time,  and 
(2)  being  able  to  neglect  the  effect  of  message  segmentation  into  pack¬ 
ets.  Hie  service  time  for  a  message  therefore  consists  of  an  average 
of  1/u  =  6.4  msec,  for  the  exponentially  distributed  message  length 
portion,  plus  an  average  overhead  ccrponent  of  XQ  =  2.9  msec,  for  the 
header  and  line  control  characters,  as  shown  in  Figure  5.1.9.  The  fig¬ 
ure  also  indicates  the  service  time  requirement  for  the  acknowledgement 
traffic  which  occurs  due  to  the  RFNM  (Request  For  Next  Message)  mechan¬ 
ism,  (i.e. ,  each  RFNM  must  be  acknowledged).  Acknowledgements  require 
a  service  time  of  X  =-3.0  msec,  for  serial  transmission,  and  occur 
with  an  average  arrival  rate,  Xg,  that  is  equal  to  the  average  arrival 
rate  of  messages,  X^,  such  that  the  ccnposite  average  arrival  rate 
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I  *, 


Figure  5.1.9  The  Composite  Service  Time  Density 


becomes: 

*  *  +  V 

and  the  oonposite  traffic  intensity  is: 


Xa  Xa  +  VX0  +  J*> 


Since  A  and  \  are  equal,  the  value  of  p  can  be  written  as: 
am  -*  c 


pc*  A 


xa  +  x0  +  Vw 


which  is  of  the  usual  form,  i.e.,  the  product  of  the  average  arrival 
rate  and  the  average  service  time. 

The  effect  of  the  acknowledgement  traffic  was  shown  in 
Equation  (4.5)  to  result  in  an  average  message  delay  of: 


W(FCFS  message  traffic) 


221 


wflere  pa  t^'e  traffic  intensity  of  acknowledqarcnts , 

Pa  =  X  Xa/2 

and  W0  is  the  expected  time  to  ccrplete  any  existing  service,  which  is 
f ran  Equation  (5.15): 

WQ  =  X  E[x2]/2 

Since  the  service  is  the  oorposite  of  two  ccnponents,  WQ  beocnes: 

w0  =  E^x  I  311  ^^1/2  +  Xm  E[x2  |  a  mes.]/2 
which  can  be  shown  to  be: 


W  -* 
w0  “  2 


2  X0  (V2+(XJ2 

(1/p)2  +  - - - 5 — 


For  the  values  of  the  service  tine  variables  given  above,  the  expres¬ 
sion  reduces  to: 


WQ  =  |[68.3] 

and  results  in  an  expected  FCFS  message  delay  of: 


W(PCFS) 


A [34.2] 
(1  -  Pa) (1  - 


For  the  p  =  0.8  case,  X  is  equal  to  0.13,  and  p 
beocnes  equal  to  0.195,  which  results  in  a  theoretical  FCFS  queueing 
delay  of  approximately  28  msec.,  compared  to  a  measured  value  of  32 
msec.  This  error  is  quite  acceptable  when  one  considers  that  a  1% 
error  in  the  arrival  rate  at  a  value  of  p  near  0.8  will  result  in  a 
queueing  delay  error  of  about  1.5  msec.  The  fact  that  the  shape  of  the 
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theoretical  and  measured  delay  curves  match  well  as  a  function  of  the 
threshold  value,  and  that  the  two  sets  of  values  agree  quite  well  for 
P  *  0.6,  gives  good  support  for  the  validity  of  the  ncdel.  The  relative 
error  for  the  p  =  0.5  case  is  surprisingly  large,  but  is  believed  to 
be  due  to  two  sources  cf  error;  (1)  the  resolution  of  the  round  trip 
time  measurement  is  0.8  risec. ,  and  (2)  the  queueing  delay  value  is  deter¬ 
mined  by  subtract  inar  the  relatively  large  fixed  portion  of  the  round 
trip  time  (about  20  msec.)  from  the  average  round  trip  measurement . 

Hence,  the  queueing  delay  values  for  the  lower  values  of  p  are  subject 
to  much  larger  relative  errors,  i.e. ,  the  same  fixed  value  error  has  a 
much  larger  apparent  effect.  More  precise  measurements  oould  have  been 
made  by  the  use  of  trace  data,  but  were  not  felt  to  be  justified  since 
differences  between  the  queueing  model  and  the  actual  system  perform¬ 
ance  are  typically  more  acute  at  the  higher  values  of  p. 

The  data  of  Figure  5.1.10  is  shown  with  the  queueing  delay 
scaled  in  msec. ,  but  can  be  normalized  by  dividing  by  the  average  ser¬ 
vice  time  of  the  composite  traffic, 

E[x  |  acrposite  traffic]  *  0. 5(3.0  msec.)  +  0.5(9. 3  msec.) 

for  an  average  value  of  6.2  msec.  Therefore,  the  normalized  queueing 
delay  for  the  p  *  0.8  case  is  4.5  canpared  to  a  normalized  value  of 
4.0  for  the  exponential  exanple  of  Figure  5.1.1.  The  acrposite  density 
has  a  larger  delay  value  due  to  the  added  delays  of  the  higher  priority 
acknowledgement  traffic,  although  the  exponential  service  density  has  a 
larger  variance  and  hence  would  otherwise  have  the  greater  queueing 
delay. 
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THEORETICAL 

EXPERIMENTAL 


0  12  3 

NORMALIZED  PRIORITY  THRESHOLD,  XT/xm 


Figure  5.1.10  A  Comparison  of  the  Theoretical  and  Experimental  Values  of 
Queueing  Delay  Versus  the  Priority  Threshold  Location 
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The  degree  of  improvement  between  the  p  *  0.8  ecses  of 
Figures  5.1.1  and  5.1.10  also  makes  an  interesting  carparison.  For  the 
exponential  service  density,  the  average  delay  was  reduced  by  42%  com¬ 
pared  to  the  PCFS  delay,  while  the  delay  reduction  for  the  composite 
service  density  was  only  30%.  This  lesser  reduction  in  delay  is  due  to 
the  fixed  service  time  cxxrponents ,  which  cause  the  theoretical  model  of 
Equation  (5.12)  to  take  on  the  form: 

w  =  w(fcfs)  1  -  *  jftl 

1  -  aX(X  +  x^] 

This  relationship  shows  that  as  the  fixed  component,  X,  is  increased 
relative  to  the  variable  component,  1/y,  the  degree  of  improvement  will 
be  decreased  accordingly.  Future  analysis  and  experimentation  should 
probably  consider  the  overhead  due  to  the  HDST-to-HOST  protocol  as  part 
of  thi3  fixed  component. 

The  general  agreement  between  the  predictions  of  the  theoreti¬ 
cal  model  and  the  experimental  data  was  very  encouraging,  but  an  even 
more  important  aspect  of  the  experiment  was  that  it  brought  about  im¬ 
provements  in  the  model.  For  example,  the  fixed  time  components  that 
accompany  each  message  had  not  been  included  in  the  original  model,  but 
became  obvious  when  the  experiment  was  being  designed.  However,  a  more 
subtle  effect  involved  the  handling  of  the  acknowledgements  for  the 
RFTM's  which  had  also  been  overlooked.  This  effect  was  uncovered  while 
attempting  to  reconcile  earlier  experimental  data,  but  was  finally  in¬ 
cluded  in  the  model  by  utilizing  part  of  the  original  priority  analysis. 
This  iterative  and  ccrplamentary  usage  of  the  measurement  and  modeling 

techniques  is  believed  to  be  a  significant  advantage  over  the  use  of 
either  approach  by  itself. 
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5.1.3 


The  Introduction  of  Additional  Priority  Classes 


For  the  case  of  priorities  being  determined  bv  an  a  prion 

knowledge  of  the  service  requirement,  we  can  generalize  the  categorize- 

fch 

tion  process  by  defining  the  1  priority  class  to  be  that  set  of 
messages  which  have  service  requirements  in  the  range,  <  x  <  x^. 

The  fraction  of  the  messages  in  the  class  is  then: 


o. 

1 


X. 

1 


b(x)dx 


xi-l 


(5.94) 


and  the  expected  service  time,  given  that  a  message  is  in  the  i 
class  is: 


x. 

X 


i  rx 1 
'sry  x 


b(x)dx 


xi-l 


(5.95) 


For  a  system  with  P  priority  classes ,  there  are  P  -  1  priority 
threshold  to  select,  subject  only  to  the  constraint  that: 


(5.96) 


This  is  a  situation  that  leads  to  an  increasingly  complex  optimization 

problem  as  the  value  of  P  becomes  larger.  To  avoid  this  combinatorial 

explosion,  we  will  only  consider  two  special  cases;  (1)  the  three 

priority  class  case  with  JL,  =  8JC,  B  >  1,  and  ( 2 )  the  case  of  equal 

12  ll 

probability  areas  for  each  class. 

5. 1.3.1  An  Example  With  Three  Priority  Classes 

A  priority  threshold,  which  divided  messages  into  two 
priority  classes  was  considered  in  Section  5.1.2,  and  we  now  wish  to 
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extend  this  notion  to  additional  priority  classes.  If  we  consider  as 

an  example,  the  case  of  two  priority  thresholds,  )C,  and  )L,  we 

L1  l2 

will  have  three  priority  classes,  and  will  define  these  classes  and  the 
associated  variables  as  shown  in  Table  5.1.1. 


TABLE  5.1.1 

NOMENCLATURE  FOR  THE  THREE  PRIORITY  CLASS  CASE 


class  1 

class  2 

class  3 

Priority 

high 

mediurt 

low 

Service  times 

x  <  \ 
ll 

<  x  <  JC, 

X1  l2 

V* 

Average  queue  delay 

W1 

W2 

W3 

Fraction  of  messages 

al 

a2 

a3 

Arrival  rate 

a^X 

°2X 

a^X 

Following  the  approach  of  Section  5.1.1,  we  have: 


W„ 


wl  = 


1  -  Xa^ 


WQ  +  XctjXjW^ 

1  -  XfctjX^  +  a^) 


W0  +  XalXlWl  +  Xa2X2W2 
W3 - W 

and  an  expected  queueing  delay  of: 

W  *  +  a3W3 


(5.97a) 


(5.97b) 


(5.97c) 


(5.98) 


227 


This  result  is  shewn  in  Figure  5.Lllfor  the  special  case  of  an 

exponential  service  density,  a  traffic  intensity  of  p  =  0.8,  and 

priority  threshold  of  3C,  =  2X_  .  The  two  priority  class  case  fran 

2  1 

Figure  5.1.12  has  been  superimposed  to  indicate  the  effect  of  the  addi¬ 


tional  priority  class,  and  as  usual,  the  axes  are  normalized  with  re¬ 
spect  to  the  mean  service  time.  The  oerparison  shows  a  relatively 
small  improvement  in  the  average  queueing  delay,  but  a  somewhat  extended 
latitude  of  threshold  values  for  which  good  improvements  were  obtained. 
The  delay  of  the  third  priority  class  shows  a  rapidly  increasing  value 

as  K_  increases,  but  of  course,  there  are  relatively  few  such  mes- 
2 

sages  at  high  value  of  for  this  distribution.  The  middle  class 

2 

messages  have  a  delay  varying  from  that  of  the  highest  priority  messages 

for  Xp  near  zero,  to  that  of  the  two  priority  case  for  large  values 

of  X  ,  since  at  both  extremes,  the  system  is  essentially  a  two  pri- 
rl 

ority  class  system.  The  highest  priority  message  delays  are  the  same 
for  both  systems  since  the  further  subdivision  of  lower  priority  mes¬ 
sages  does  not  affect  the  class  1  messages. 

In  the  above  example,  the  two  priority  thresholds  were  related 

by  Xp  =  2Xp  .  We  can  expand  this  relationship  by  introducing  the 
2  A1 

parameter,  8  such  that: 


The  effect  of  varying  this  parameter  is  illustrated  in  Figure  5.1.12 
for  a  hyper exponent ia 1  service  density,  and  for  a  traffic  intensity  of 
p  =  0.6.  Note  that  for  8=1,  the  two  thresholds  are  superimposed, 
and  degenerate  to  the  two  priority  class  case.  The  improvement  for 
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Figure  5.1.11,  A  Comparison  of  th«  Daisy  Curves  for  Two  and  Three 
Priority  Classes.  (Exponential  service) 
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6-2  is  similar  to  that  observed  for  the  exponential  density  in 
Figure  5.1.11  and  a  slight  additional  improvement  is  obtained  for 
8  =  4.  However,  for  higher  values  of  t,  no  further  decreases  were 
observed,  and  the  curve  begins  to  revert  to  the  6=1  case  which  it 
will  approach  asymptotically  as  6  becomes  infinitely  large. 

One  should  not  be  too  dogmatic  about  conclusions  drawn  fran 
these  two  exarrples ,  but  they  seem  to  indicate  that  additional  priority 
classes  provide  little  improvement  in  the  queueing  delay,  but  do  in¬ 
crease  the  latitude  of  the  improvement  relative  to  the  threshold  loca¬ 
tion.  That  is,  the  precise  threshold  location  is  not  as  critical  for  a 
given  service  density,  and  conversely,  a  given  set  of  thresholds  may  be 
reasonable  for  a  wider  class  of  service  density  functions. 

5.1. 3.2  The  Analysis  for  P  Priority  Classes 

The  queueing  delay  expressions  of  Equation  (5.97  a-c)  can  be 
readily  extended  to  the  case  of  P  priority  classes  by  first  rewriting 
them  in  terms  of  W^, 

W0 

W  =  - (5.99a) 

1  -  Aa^ 

w 

W  =  - - - ^ - - - - -  (5.99b) 

(1  -  [1  -  Ata^  +  c^Xj)] 

W0 

w.  =  - - - - - y - - - - - - -  (5.99c) 

(1  -  Ata^  +  a^)]*!!  -  AtajX^  +  +  a^)] 

and  noting  that  the  results  are  of  the  form: 
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(5.9<W) 


W0 


p-1 

p 

1  -  x  E  Vi 

1  - x  £  °jxl 

which  is  Cobham's  result  (C054) .  We  are  more  concerned  with  the 
expected  queueing  delay  averaged  over  all  message  classes, 


W  =  +  0^2  +  . . .  +  OpWp 


(5. inn) 


which  after  considerable  algebra,  can  be  represented  in  the  form  of  the 
PCFS  delay  and  an  improvement  factor, 


This  is  the  generalization  of  Theorem  5.1.2  to  the  P  priority  class 
case,  and  has  the  same  generality  of  priority  classification  as  the 
original  theorem.* 

If  one  expands  the  products  of  Equation  (5.101),  the  numerator 
and  denominator  cancel  except  for  two  factors  leaving, 


W  = 


W„ 


a  (1  -  p) 


r~l  Lj 

p-i 

P 

p=i 

i  - x  E  «ixi 
i=l  1  1 

• 

i  -  >'  %  Vi 
3*1  3  3 

(5.102) 


in  which  the  1  -  p  factors  cancel  leaving  a  result  given  by  Cox  and 
9rdth  (C061) .  However,  the  cfjicellation  of  the  1  -  p  factor  in  their 


IF - 

Due  to  the  generality  of  the  priority  classification  of  Equation  (5.101) 
the  factor  within  the  braces  will  not  necessarily  be  less  than  unity. 
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previous  analysis  lost  the  more  explicit  and  intuitive  form  or  repre¬ 
senting  the  expected  delay  as  the  product  of  the  expected  PCFS  delay 
and  a  reduction  factor  as  shewn  in  Equation  (5.101) . 

5. 1.3. 3  The  Limiting  Case  of  a  Continue  of  Priority  Classes 

If  the  number  of  priority  classes  is  allowed  to  increase 
indefinitely,  the  surma tion  of  Equation  (5.102)  approaches  the  integral 
result  of  Phipps  (PH56) , 


W  = 


(5.103) 


where  the  expected  remaining  service  time,  Wg,  is  replaced  by  its 
equivalent. 


7/2 


and  the  a  's  become  the  elemental  areas  dB(x) ,  or  in  the  case  of 
n 

continuous  functions,  b (x)dx. 

If  we  consider  the  hyper exponential  service  time  example. 


b (x)  «  0.5  u  e-MX  +  0.5  y  y  e 
then  we  know  from  Equation  (5.45)  that: 

?=  (i  +  y2)/(m  y)2 

and  one  can  show  that: 


f\  btyiay  = 


(5.104) 
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where: 


iMux)  -  U  -  (1  +  ux)exp(-ux)]  +  ^  [1  -  (1  +  yux)exp(-yux) ) 
The  integral  for  the  expected  delay  then  beam's: 

U  E {queue  delay]  =  ~  •  ■— y  f -P S“) —  d*  (5.105) 

Y  “  ii  -  $ 

Due  to  the  oorplexity  of  the  integral  when  we  include  the  expressions 
for  b(x)  and  t{/(ux),  the  evaluation  was  performed  numerically  to  ob¬ 
tain  the  results  shewn  in  Figures  5.1.13  and  5.1.14.  The  first  figure 


p  - 


Figure  5.1.13.  A  Comparison  of  the  Queueing  Delays  for  the  Cases  of  No 
Priorities,  Two  Priority  Classes,  Three  Priority  Classes,  and 
a  Continuum  of  Priority  Classes 
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HYPEREXPONENTIAL 
y  -  1/3,  p  -  0.6 


Figure  5.1.14.  Typical  Plot*  of  Quauaing  Daily  Varus  Priority  Thrashold 


is  a  coitparison  of  the  optimal  delay  values  for  the  cases  of  two  and 
three  priority  classes,  with  the  FCFS  delay  and  the  continuous  priority 
delay  shown  as  bounds  on  the  delays.  The  introduction  of  a  single  pri¬ 
ority  threshold,  i.e.,  two  priority  classes,  can  be  seen  to  acccrplish 
the  major  portion  of  the  available  reduction  in  delay,  but  one  should 
recall  that  the  latitude  of  reasonably  optimal  values  was  also  improved 
by  further  priority  classifications.  In  cases  where  the  density  func¬ 
tion  is  unknown  or  is  variable,  this  increase  in  latitude  may  be  par¬ 
ticularly  important.  Figure  5.1.14  shews  a  typical  plot  of  the  delay 
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versus  the  threshold  value  and  includes  the  continuous  priority  model 
as  the  lower  brxind.  Here  again,  one  can  see  the  closeness  to  this 
limiting  case  for  the  case  of  merely  two  or  three  priority  classes. 

5. 1.3. 4  Considerations  of  the  Cost  to  Inplement  an  Additional 

Priority  Class 

The  optimal  priority  system  was  shewn  to  he  one  with  a 
oontinuun  of  priority  classes.  However,  if  one  associates  a  non-zero 
cost  to  implementing  the  priority  classifications,  the  optimal  value 
will  be  for  seme  finite  number  of  priority  classes,  and  we  will  next 
(Consider  the  problem  of  how  to  find  that  optimal  number  of  classes. 

One  of  the  problems  in  the  optimization  of  multiple  priority 
classes  is  the  selection  of  the  various  priority  thresholds,  i.e. ,  the 
borders  between  the  classes.  We  shall  arbitrarily  resolve  this  by 
selecting  equal  probability  areas  for  the  priority  classes.  For  ex¬ 
ample,  if  one  has  P  different  priority  classes,  the  boundaries  shall 
be  selected  such  that: 


/ 


b(x)dx  =  1/P  for  i  =  1,2,. ..,P 


(5.106) 


Y 


i-l 


where: 


JC,  =  o"  and  \ 

0  P 


Cobh  an 's  result  which  was  derived  in  Section  5. 1.3. 2  states  that: 


W„ 


W_ 


P-1 

P 

1  ■ x  X  aixi 

i«l  1  XJ 

1  '  *  ajXj. 

,  p  =  1,2,. . . ,P  (5.107) 
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Where: 


W 


=  E (queue  delay  for  a  priority  class  p  message] 


and  the  priority  precedence  relations  are  such  that  class  1  messages 
have  priority  over  class  2  messages,  etc.  The  average  queue  delay  is: 


W  =  0^1  +  0^2  +  . . .  +  otpWp 


(5.108) 


but  for  the  case  of: 

~  <*2  •  •  •  —  ®p  —  1/^* 


then: 

W  -  i  (Wx  +  W2  +  ...  +  Wp]  (5.100) 

with  each  delay  term,  W^,  being  as  given  in  Equation  (5.107).  A  plot 
of  the  delay,  W,  versus  the  number  of  priority  classes  is  shewn  in 
Figure  5.1.15  for  severed  values  of  p  and  an  exponential  service  den¬ 
sity.  Once  again  the  rapidity  with  which  the  delay  reduces  as  the  first 
few  priority  classes  are  introduced  is  apparent.  The  figure  also 
shows  the  delay  values  for  the  optimal  two  priority-class  case,  which 
for  p  equal  to  0.8,  results  in  a  further  delay  reduction  of  about  10% 
beyord  that  of  the  "equal  probability  area"  threshold. 

If  we  include  a  cost  function  to  account  for  the  overhead 
associated  with  the  priority  mechanism,  we  must  establish  two  criteria. 
First  the  unit  of  cost  must  be  selected,  e.g.,  in  actual  dollar  costs, 
in  CPU  utilization,  in  added  message  delay,  or  perhaps  in  terms  of  mem¬ 
ory  requirements;  and  secondly,  the  functional  relationship  between  the 
cost  and  the  number  of  priority  classes  must  be  approximated.  An 
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Figure  5.1.15.  The  Effect  of  the  Number  of  Priority  Clones  on  the 
Expected  Queueing  Delay 
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investigation  of  several  possible  priority  implementation  schemes 
indicated  that  the  most  critical  cost  effect  was  the  core  storage  re¬ 
quirement  for  the  priority  determination  and  marking  routines,  arri 
the  additional  linked  lists  to  mechanize  the  priority  queues.  These 
costs  grow  in  a  manner  which  we  will  approximate  by: 

cost  «  K  ln(N)  (5.110) 

where  N  is  the  number  of  priority  classes  and  K  is  a  free  variable 
to  be  selected.  The  choice  of  the  base  for  the  logarithm  is  arbitrary, 
since  it  merely  changes  the  value  of  K,  but  for  convenience  in  hand¬ 
ling  the  integer  values  of  N,  we  will  utilize  base  jwd  logarithms. 

The  cost  function  then  becomes; 

cost  (in  memory  requirements)  =  K'  log2  (N)  (5.111) 

where  K'  is  equal  to  the  memory  requirement  to  imple:oent  the  two 
priority  class  case,  which  we  will  refer  to  as  The  cost  function 

to  implement  P  priority  classes  can  then  be  written  as: 

Cp  =  Cost  (in  memory  requirements)  =  Mjlog^fP)  (5.112) 

and  is  the  cost  that  must  be  considered  when  one  implements  additional 
priority  classes.  The  logarithmic  cost  function  was  chosen  for  a 
variety  of  reasons  including;  (1)  it  has  a  zero  value  for  a  single 
priority  class,  i.e.,  PCFS,  (2)  it  has  a  decreasing  slope  as  the  number 
of  priority  classes  is  increased  (which  accounts  for  the  use  of  more 
efficient  priority  queue  handling  mechanisms) ,  and  (3)  because  the  time 
required  for  the  priority  classification  via  a  binary  search  is  kncwn 
to  be  proportional  to  the  logarithm  of  the  number  of  categories. 
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If  we  assure,  as  in  Section  5. 1.2. 4,  that  messages  have  sore 
predefined  maximal  length,  L,  which  defines  the  block  size  for  storage 
allocation,  then  on  the  average,  n  such  blocks  will  be  linked  to- 
gether  on  an  output  queue,  where  by  Little's  result: 

Efnutber  in  queue]  =  n  =  AW 

If  each  such  storage  block  is  of  length  L,  then  the  expected  storage 
requirement  for  the  enqueued  messages  is: 

E[storage  for  buffering]  =  AWL  (5.113) 

The  requirement  that  message  lengths  have  seme  such  finite  upper  bound 
is  a  practical  necessity,  but  does  not  introduce  any  theoretical  prob¬ 
lem  in  the  model  since  the  message  length  distributions  can  include  a 
discrete  (delta  function)  ccrponent  at  the  length,  L.  Under  these  con¬ 
ditions,  the  expected  storage  requirement  for  buffering  can  be  reduced 
by  the  same  priority  techniques  that  reduce  the  average  queueing  delay, 
W,  and  when  both  the  priority  implementation  and  buffering  considera¬ 
tions  of  Equations  (5.112)  and  (5.113)  are  combined,  the  total  storage 
requirement  for  a  s tore- and- f orward  node  with  Q  output  queues  becomes: 

E(storage]  =  QALW(P)  +  f^log^P)  (5.114) 

If  it  were  not  for  the  M2log2(P)  component,  this  expected  value 
would  be  a  continually  decreasing  function  of  the  number  of  priority 
classes.  However,  the  grazing  implementation  cost  component  will  re¬ 
sult  in  a  finite  optimal  number  of  priority  classes,  and  it  is  this 
number  of  priority  classes  which  we  wish  to  determine. 

The  example  of  Figure  5.1.15  was  extended  to  include  this 
cost  function,  losing  the  approximate  implementation  costs  of  the  ARPA 
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network  IMP,  i.e.,  a  requirement  of  about  25  IMP-words  to  irrplenont  the 
two-priority  class  mechanism,  and  L  =  63  IMP-words,  for  a  value  of 
M2  of: 

M2  =  ||  L  =  0.4L 

Hie  expression  for  the  average  storage  requirement  is  then: 

E [storage]  =  QXx  L  +  h.4L  log,(P) 
x  1 

which  can  be  written  as: 

E [storage]  =  QpL  yW(P)  +  Q.4L  log2(P)  (5.115) 

This  function  is  plotted  in  Figure  5.1.16  for  the  exponential  service 
example  of  Figure  5.1.15  and  for  Q  =  3.  Several  values  of  p  are 
shewn,  with  definite  optimal,  i.e.,  minimal  cost,  values  being  indi¬ 
cated.  As  expected,  the  higher  values  of  p  show  the  greatest  im¬ 
provement  due  to  the  priority  implementation,  and  show  the  benefit  of 
introducing  additional  priroity  classes,  up  to  a  maximum  of  about  three 
or  four  classes.  In  contrast,  the  lower  values  of  p  indicate  that 
the  FCFS  system  has  the  minimal  cost  since  the  average  number  of  mes¬ 
sages  in  the  queue  is  very  small.  However,  the  effects  are  of  primary 
concern  at  the  higher  values  of  p,  since  effective  buffer  utilization 
is  more  critical  under  heavy  loading. 

5*1*3. 5  Design  of  an  Experiment  to  Verify  the  Multiple  Priority  Model 

An  experiment  could  be  run  to  verify  the  multiple  priority 
model  by  an  approach  quite  similar  to  that  utilized  in  Section  5. 1.2. 5 
for  the  two  priority  model.  Artificial  traffic  would  be  set  up  as 
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Figure  5.1.16  The  Expected  Storage  Requirements  Including  the  Priority  Implementation  Costs 
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before  using  fixed  length  messages  and  varying  the  number  of  active 
generators  to  establish  the  desired  levels  of  the  traffic  intensity,  p. 

For  purposes  of  the  design  of  a  suggested  experiment,  it  will 
be  assumed  that  a  multi-priority  system  will  be  investigated  using  the 
equal  probability  area  criterion  for  determining  the  priority  classes. 
An  exponential  distribution  of  message  lengths  would  be  utilized  with 
an  average  length  of  40  bytes  (to  minimize  the  truncation  effect  of 
packet  segmentation) .  Therefore ,  the  priority  threshold  for  the  two 
priority  class  system  would  be  set  at  a  length,  L^,  where: 

e~Vr=0.5 

with  T  equal  to  20,  i.e.,  the  average  message  length  in  terms  of 
16-bit  IMP  words.  (The  priority  threshold  should  be  expressed  in  terms 
of  IMP  words  since  that  is  the  way  in  which  the  variations  would  be 
made  during  the  experiment.)  For  the  two  priority  class  system  the 
priority  threshold  is  therefore, 

L  =  0.69r  =  14  words 
P 

This  value  will  be  sub-optimal  because  the  optimal  threshold  is  known 
to  be  greater  than  the  mean  value.  However,  it  should  serve  as  an 
experimental  test  point  to  compare  with  the  theoretical  results  of 
Figure  5.1.15.  The  thresholds  for  the  three  priority  case  can  be  de¬ 
termined  in  a  similar  manner  by: 

e’Lpi/  r  =  0.67 
and 
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with  resulting  threshold  values  of: 

Lp  =  0.4(K  =  8  words 

and 

L  =  l.lll  =  22  words. 
p2 

The  thresholds  for  higher  nuntoers  of  priority  classes  would  be  deter¬ 
mined  in  a  similar  manner,  with  experimental  data  taken  for  each  such 
set  of  values  and  compared  with  the  theoretical  results  of  Figure 
5.1.15.  Values  of  p  equal  to  0.4,  0.6,  and  0.8  would  be  adequate  to 
verify  the  model,  and  should  give  large  enough  queueing  delays  to  give 
good  results  from  the  round-trip  delay  portion  of  the  accumulated 
statistics.  The  UCIA-to-RAND  path  would  again  be  utilized  for  the  test 
to  avoid  alternate  routing  effects. 

The  largest  error  for  such  a  test  should  occur  for  the  highest 
value  of  p,  which  for  p  equal  to  0.8  ±  0.05,  would  result  in  errors 
of  about  ±20%  to  30%  about  the  nominal  delay  value.  The  accuracy  of 
creating  a  traffic  level  of  p  equal  to  0.8  should  be  somewhat  better 
than  this  ±0.05  value,  so  that  the  error  would  not  be  expected  to  be 
quite  that  large,  but  could  easily  be  in  the  10%  to  15%  region. 

This  experiment  was  not  performed  due  to  the  difficulty  of 
implementing  the  various  priority  mechanisms  by  IMP  program  changes,  and 
the  fact  that  such  changes  would  have  to  be  performed  by  the  network 
contractor,  BBN. 
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5.2 


Models  for  the  Segmented  Message  Case 


The  preceding  analysis  assumed  that  messages  would  be 
serviced  in  a  contiguous  manner,  and  that  message  buffers  were  of  suf¬ 
ficient  length  to  hold  a  maximal  size  message.  A  nurber  of  practical 
desires  and  constraints  can  not  neoessarily  be  met  by  such  a  scheme, 
and  therefore  most  message  switching  systems  are  designed  to  handle 
fixed  length  message  segments.  Sane  of  the  consequences  of  such  seg¬ 
mentation  are  considered  in  the  following  sections. 

5.2.1  Rationale  and  Effects  of  Segmentation 

The  wide  variation  in  possible  message  lengths  leads  to  the 
segmentation  of  messages  into  shorter  fixed  length  blocks  or  packets, 
to  smooth  the  service  demands,  and  to  give  better  response  to  short 
interactive  messages.  The  buffer  allocation  problem  may  also  be  af¬ 
fected  by  the  segmentation  because  of  the  need,  in  most  equipment,  to 
allocate  space  in  fixed  size  blocks,  but  such  segmentation  does  not 
necessarily  affect  the  message  flew  since  snail  buffers  can  be  linked 
together  to  store  a  given  length  message.  However,  if  the  messages 
must  be  transmitted  as  a  contiguous  bit  stream,  interactive  messages 
may  be  delayed  excessively,  so  that  seme  type  of  preemptive  or  pre¬ 
empt!  ve-resune  discipline  might  be  considered.  The  segmentation  of 
messages  into  packets  is  similar  to  the  preemptive- res une  situation, 
but  preemption  is  allowed  only  at  the  packet  boundaries.  Sane  overhead 
is  required  in  either  case,  with  the  individual  packet  overhead  being 
the  connuni cation  control  characters  and  the  header  information.  The 
advantages  of  segmenting  messages  into  packets  generally  favor  small 
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packet  sizes,  but  the  packet  overhead  cost  must  be  balanced  against 
these  gains  to  determine  the  optimal  packet  size.  We  will  consider 
some  of  these  packet  size  selection  criteria  in  Section  5.2.4,  but  for 
no/  will  further  explore  the  advantages  of  reducing  the  packet  size. 

The  analysis  of  Section  5.1.1  shewed  that  the  expected 
queueing  delay  is  proportional  to  the  average  time  to  complete  an  ex¬ 
isting  service,  Wg,  which  in  turn  is  proportional  to  the  second  rrement 
of  the  service  time.  Truncation  of  the  service  time  density  reduces 
this  second  moment,  as  shorn  in  Appendix  A,  and  thereby  leads  to  shorter 
queueing  delays  for  priority  messages. 

A  second  effect  of  interest  involves  the  changes  in  the 
arrival  times  when  messages  are  segmented.  By  definition,  an  arrival 
occurs  when  the  last  bit  of  a  contiguous  bit  stream  reaches  the  IMP, 
and  the  effect  of  the  different  arrival  epochs  in  reducing  the  amount 
of  unfinished  work  in  the  system  is  shewn  in  Figure  5.2.1.  *ie  seg¬ 
mented  message  case  also  decreases  the  total  delay  by  the  fact  that 
arrivals  occur  earlier,  and  that  the  input  and  output  processes  tend  to 
be  overlapped.  The  reduction  in  the  amount  of  unfinished  work  implies 
that  the  storage  requirement  should  also  be  less  for  the  segmented 
case,  a  consideration  that  will  be  pursued  in  Section  5. 2. 3.1. 

A  third  consequence  of  segmentation  is  its  effect  on  the 
error  control  techniques  used  to  detect  and  correct  transmission  errors. 
Longer  segments  are  more  likely  to  contain  randan  errors  and  also  make 
retransmission  more  costly,  but  allow  the  use  of  more  efficient  error 
control  codes.  An  optimal  balance  of  these  factors  depends  on  the 
error  mechanism  and  rates  involved,  and  has  been  considered  in  previous 
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(a)  UNFINISHED  WORK  DIAGRAM  FOR  CONTIGUOUS  MESSAGES 


(b)  UNFINISHED  WORK  DIAGRAM  FOR  SEGMENTED  MESSAGES 


Fiflura  5.2.1.  Typical  Unfinished  Work  Diagrams  for  Contiguous  and  Segmented  Message  Flows. 
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studies  including  that  of  Kirlin  (KI69) .  The  line  quality  and  error 
control  in  the  ARPA  network  have  been  quite  effective  and  do  not  appear 
to  lx?  a  determining  factor  for  packet  size,  and  therefore  will  not  lie 
considorrxl  further.  (A  24-l>it  cyclic  error  chocking  oulo  i;;  . i| g « nx  1< xl 
to  each  packet,  and  is  checked  and  regenerated  at  each  store- and- 
forward  IMP) . 

The  segmentation  of  messages  into  separate  packets  which  can 
flew  through  the  network  independently  introduces  the  possibility  of 
gaps  between  packet  transmissions.  These  gaps  are  typically  due  to 
other  traffic  in  competition  for  the  service  facility,  and  cause  addi¬ 
tional  delay  components  at  the  destination  because  the  message  must  be 
fully  reassembled  prior  to  transmission  to  the  HOST.  Aside  from  such 
gaps  between  packets,  the  individual  packets  may  have  arrived  out  of 
sequence,  duplicate  packets  may  have  been  recieved,  or  in  rare  cases 
one  or  more  packets  may  have  been  lost,  so  that  the  reassembly  process 
requires  numbering  the  packets  and  indicating  the  final  packet  of  a 
message,  as  well  as  time-outs  to  discard  message  fragments  in  the  latter 
case.  The  selection  of  this  time-out  vawe  and  the  mechanism  for  allo¬ 
cating  reassembly  buffers  are  important  system  parameters,  and  need 
careful  attention.  Several  possible  allocation  procedures  are  possible 
including;  (1)  reserving  the  proper  number  in  advance  by  a  "request  to 
send",  (2)  setting  aside  a  full  set  of  message  reassembly  buffers  upon 
the  receipt  of  the  first  packet  of  a  message,  and  (3)  allocating  buffers 
on  an  "as  arrived"  basis.  A  concern  in  the  latter  case  is  that  in 
periods  of  congestion,  few  complete  messages  might  be  reassembled  due 
to  the  exhaustion  of  the  buffer  supply  by  numerous  partially  reassembled 
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messages . 


The  area  of  message  segmentation  is  rich  in  possible 
advantages  and  pitfalls,  which  makes  it  a  viable  area  of  analysis  and 
measurement,  with  the  goal  of  inproving  the  network  performance  by 
finding  more  optimal  techniques  and  parameter  values.  Severed  such 
aspects  are  considered  in  the  following  sections,  with  both  analytic 
and  experimental  methods  being  utilized. 

5.2.2  The  Separation  Between  Packets 

If  there  were  no  other  traffic  in  a  store-and-forward  network, 
the  packets  of  a  message  would  flow  through  the  net  in  a  continuous 
stream.  However,  when  other  traffic  is  introduced  at  the  various  nodes, 
the  various  message  packets  tend  to  be  acme  interspersed,  and  the  packets 
of  any  given  message  may  beocme  separated.  This  intermixing  does  not 
necessarily  change  the  average  delay  in  flowing  through  the  network, 
since  the  messages  are  essentially  time-sharing  the  channel  capacity, 
aid  analogous  tine-shared -computer  analysis  results  have  indicated  cer¬ 
tain  conservation  relationships  (KL64).  However,  the  network  problem 
differs  in  several  ways  including  the  lack  of  well  ordered  service 
scheduling,  and  also  in  the  fact  that  the  net  involves  a  sequence  of 
such  facilities.  There  is  an  intuitive  feeling  that  as  packets  beocme 
separated,  interference  may  beocme  more  likely  and  therefore  that  the 
expected  gap  size  may  tend  to  grow  as  the  message  flews  through  a  large 
network.  We  will  show  in  Theorans  5.2.1  and  5.2.2  that  the  expected 
gap  size  is  bounded  for  both  the  FCFS  and  priority  disciplines  respec¬ 
tively.  Interestingly,  this  bound  is  the  same  in  both  cases,  and  is 
equal  to: 
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lim  E  [interpacket  gap  after  traveling  thourgh  n  nodes] 

n-x» 


where  is  the  traffic  intensity  of  the  interfering  traffic. 

Theorem  5.2.1 

The  expected  interpacket  gap  for  a  segmented  message,  and  a 
FCFS  queue  discipline,  is: 

E[t  I  =  [1  -  (Pi>n_1]  •  Xs  (5.117) 

1 

where  xn  is  the  extra  interpacket  spacing  at  the  output  of 
th 

the  n  node  due  to  interfering  traffic  along  a  store-and- 
forward  path,  Xs,  is  the  service  time  of  a  maximal  length 
packet,  and  p,.  is  the  traffic  intensity  in  terms  of  the 
average  arrival  rate  of  interfering  packets  and  their  service 
requirements .  All  p^'s  are  assured  to  be  equal  along  the 
entire  network  path.  Interfering  packet  arrivals  at  each 
node  are  also  assumed  to  be  Poisson,  with  parameter  A^. 

Proof: 

If  we  consider  a  "tagged"  multipacket  message  at  the  originating  node, 
consisting  of  packets  we  find  a  situation  as  shown 

in  Figure  5.2.2(a).  The  packets  are  presented  to  the  queue  as  a  group, 
and  for  the  FCFS  system,  are  transmitted  continguously.  The  arrivals  at 
the  second  nude  are  therefore  spaced  at  regular  intervals  of  T^  =  Xs 
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Figure  5.2.2.  The  Effect  of  Interfering  Traffic  to  Create  a  Separation  Between  Arrivelt  of  Packets  of 
a  Message. 
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seconds,  where  Xg  is  the  time  required  to  service  one  maximal  lenqth 
packet.  However,  at  this  second  node  there  may  be  interfering  traffic 
arriving  on  other  input  channels  with  an  arrival  rate  of  A^  and  as 
shewn  in  Figure  5.2.2(b),  these  packets  will  become  interspersed  with 
the  packets  of  the  tagged  message.  The  expected  value  of  the  time  be¬ 
tween  the  tagged  packet  departures  will  be: 

T  =  X  +  7-  (5.118) 

2  S  2 

where  the  gap  between  the  now  non-contiguous  packets  will  have  an 
expected  value  of: 

T 2  =  n^2  x.  (5.119) 

with  the  subscripts  i  referring  to  the  interfering  traffic  and  the 
subscript  2  indicating  that  this  delay  is  at  the  output  of  the  second 
node  along  the  path.  The  expected  value  2  is  the  average  number  of 
interfering  arrivals  during  the  interarrival  time  between  the  tagged 
packets,  which  for  Poisson  interference  arrivals  is: 

n.„  =  A.  X  (5.120) 

1.Z  1  s 

and  the  value  is  the  average  service  requirement  of  the  interfering 
traffic,  such  that: 

p.  =  A.  x.  (5.121) 

i  ii 

The  expected  gap  between  the  packets  at  the  output  of  the  second  node 
can  then  be  written  as: 

E[t21  =  Pi  Xg  (5.122) 
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and  this  gap  becomes  the  interarrival  gap  at  the  third  node.  There  is  a 
non-zero  probability  that  the  interfering  packets  which  separate  the 
tagged  packets  will  be  routed  to  the  sane  output  channel  at  the  third 
node,  but  we  shall  ignore  this  effect  and  assume  that  only  new  arrivals 
will  be  contending  for  the  channel.  Interfering  traffic  can  occur  any¬ 
time  during  the  interarrival  time  which  has  now  grown  to  X  +  t,  such 
that  the  expected  number  of  interfering  arrivals  becomes: 

ni3  =  Ai(Xs  +  t2)  (5.123) 

and  the  resulting  gap  between  the  tagged  packets  at  the  output  of  the 
third  node  becomes: 

E[t3]  =  \L(Xs  +  E[r2])xi 

or  substituting  for  E[t2]  ,  and  simplifying: 

E[t3]  =  Pi(l  +  pi)Xs  (5.124) 

A  similar  line  of  reasoning  at  the  fourth  node  produces, 

E[t4]  =  Pi(xs  +  E[t3]) 
or: 

E[t4]  =  pi(l  +  pL  +  P^JXg 

and  leads  to  the  general  iterative  expression: 

E[xn]  =  p.(Xs  +  EIt^])  (5.125a) 

or: 


n-2 


E[Tn!  = 


X.  E 


s  «,«»!»■ 
k=0 


(5.125b) 
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Equation  (5.125)  can  bo  expressed  in  closed  form  as: 


E[rnl 


°ixs 


and  by  grouping  terms: 


EtTn]  =  tl  -  (Piln’1l(r4^“)xs 

with  xn  being  the  gap  between  a  pair  of  tagged  packets  at  the  output 
of  the  n*"*1  store- and-f orward  node.  This  corpletes  the  proof  of 
Theorem  5.2.1. 


The  expected  interpacket  gap  is  shown  in  Figure  5.2.3  for 

several  values  of  p  ,  and  indicates  several  interesting  aspects  about 

i 

the  gap  size.  For  relatively  small  values  of  p^,  the  expected  gap 
length  reaches  an  asymptotic  value  after  going  through  a  few  nodes,  with 
larger  values  of  p^  requiring  long  paths  to  reach  a  similar  asymptote. 
The  fact  that  in  each  case  an  asymptote  exists  is  of  interest,  since  it 
shews  that  the  gap  does  not  grew  in  an  unbounded  fashion,  but  of  par¬ 
ticular  interest  is  the  mathematical  form  of  this  asymptotic  value,* 


pi 

lun  E[t  ]  =  y-— 
n* *  n  1  -  f 


(5.126) 


It  is  interesting  to  note  that  for  the  expected  interpacket  gap  to  be¬ 
came  within  a  factor,  F,  of  the  asymptotic  value,  x"  ,  we  need: 


n  >  1  + 


log(l  -  F) 
log  PL 


This  form  is  believed  to  be  analogous  to  the  conservation  results  of 
Kleinrock  (KL64)  for  various  time-sharing  queue  disciplines. 
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X,-  SERVICE  TIME  FOR  A  FULL  PACKET 


n 


Figure  5.2.3.  Expected  Inter-Packet  Seperetion  After  Going  Through  n  Nodes. 


This  can  be  shewn  by  noting  that  the  expression  for  E[t  ]  can  be 

n 

written  as: 


E[t  ]  =  [1  -  (p.)n_1]  t"  =  F  t" 

n  l  oo  oo 


such  that: 


F 
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and  by  taking  logarithms  of  both  sides, 

+  log  (1  -  F) 
leg  P£ 

Therefore ,  any  value  of  n  equal  to  or  greater  than  this  value  will 
cause  the  expected  gap  to  be  within  the  factor ,  F ,  of  the  asymptotic 
delay  value. 

An  extension  of  interpacket  gap  model  was  obtained  by  a 
colleague*  who  considered  the  case  of  non-uniform  traffic  intensities 
along  the  store— and— forward  path.  This  change  meant  that  the  expected 
interpacket  delay  would  not  necessarily  increase  at  each  subsequent 
node,  resulting  in  an  extension  of  the  iterative  relationship  of 
Equation  (5.125a)  to  include  this  case, 


E[Tn]  =  max{pin(Xs  +  E{Vl)  ,  E[Vin 


(5.127) 


where  p.  is  the  traffic  intensity  of  the  interfering  packets  at  the 
in 

n^1  node. 

The  interpacket  gap  analysis  is  a  priority  system  differs  in 
two  aspects  from  that  of  the  FCFS  system.  In  the  FCFS  system,  any 
arrivals  occurring  between  the  arrivals  of  packets  Pj  and  '  *"* 

also  serviced  between  Pj  and  Pj+^.  Pri°ri^y  system,  the 

interfering  arrivals  will  be  any  non-priority  packets  between  the 
arrivals  of  Pj  and  Pj+1,  and  any  priority  arrivals  after  Pj  begins 
service  but  before  Pj+1  begins.  The  resulting  interpacket  gap  given 


*This  extension  was  made  by  Gary  Fultz,  who  was  concerned  with  differ¬ 
ing  traffic  intensities  across  the  net  in  routing  simulation  studies. 
This  interpacket  gap  tirre  was  of  concern  to  him  because  of  its  effect 
on  the  reassembly  time  of  messages  at  the  eventual  destination. 
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(a)  THE  QUEUE  AND  SERVER  ACTIVITY  AT  THE  SOURCE  NODE 


Pi  pj  +  1 


SERVER  - 

T 

M 

X 

1 

T2  |“* - *S - "“j 

QUEUE  ■"  _ _ 

T 

j— — ^ 

lPj  +  1 

1 

1  ! 

Pi  ♦ 

U - \ 

1  — *1 

7,' NON  PRIORITY  v2  PRI0RITY 

ARRIVALS  ARRIVALS 


(b)  THE  QUEUE  AND  SERVER  AT  THE  SECOND  NODE 


Figure  5.2.4.  Arrivals  and  Departures  for  the  Queue  and  Server  in  the  Priority  Queueing  Case. 
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in  Theorem  5.2.2  is  shown  to  increase  more  rapidly  than  that  of  the 
non-priority  case,  but  to  have  the  same  limiting  value. 

Theorem  5.2.2 

The  expected  interpacket  gap  for  a  segmented  message  with 
higher  priority  traffic  competing  for  the  server  is: 


(5.128) 


where  the  conditions  and  variables  are  as  defined  in 

Theorem  5.2.1  with  p„  and  p  being  the  traffic  inten- 

P  rip 

sities  of  priority  and  non-priority  traffic  respectively. 


Proof: 


Assume  that  a  multipacket  message  is  initially  entered  into  the  network 

A.U  — 1. 

as  shewn  in  Figure  5.2.4(a).  The  y  and  j+1  packets  of  the 
message  are  not  necessarily  served  contiguously  (as  in  the  PCFS  case) 
since  priority  arrivals  during  the  service  of  will  be  serviced 
before  P^+1  as  will  other  priority  arrivals  during  the  service  of  the 
priority  messages.  If  the  arrival  rate  of  interfering  priority  messages 
is  then  the  expected  time  between  the  departures  of  Pj  and 


Pj+1  is: 


E[TX]  =  Xs  +  E^] 


(5.129) 


where  is  the  time  to  service  the  priority  arrivals,  with: 


EfvlJ  =  Xip(Xs  +  E[T1J) 


(5.130) 
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and  since  the  arrivals  and  service  requirements  are  independent, 


E [t1]  =  E[v^l  •  E[x  |mes.  is  priority] 


or: 


E[T1]  =  Xip(Xs  +  E[T1I)X1 


(5.131) 


Solving  fur  E[x^]  we  obtain: 


E[t ,  .  meAI.  .  JeA 

-L  i  a  7.  -L  P  . 


(5.132) 


1  -  x. 
ip  1 


where  p  is  the  traffic  intensity  for  interfering  priority  messages, 
P 

p  -  x. 

P  ip  1 


'Ihe  expected  interdeparture  time  of  packets  and  P^  at  the 
first  node  is  then: 


Eri]  =  Xs  +  l*= 


X 


1  "  P_ 


(5.133) 


so  that  at  the  second  node  we  can  expect  to  find  n  interfering 
arrivals  of  regular  messages,  where: 


E[n]  =  X^p  EtTJ  =  Xlnp(Xs  +  E[Tll)  (5.134) 

Only  the  non-priority  messages  are  considered  to  be  interfering  in  the 
sense  of  adding  to  the  interpacket  gap.  Any  priority  arrivals  are 
either  served  prior  to  P .  or  are  considered  in  the  v-  interfering 

J 

priority  arrivals,  where  v->  has  an  expected  value  of: 

E[v21  -  xip<xs  +  E[t21  )  (5.135) 
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If  we  again  assume  an  independence  of  service  and  arrivals,  the 
expected  interdeparture  time  becomes: 


E[T2]  =  xs  +  E[n1]  *  x2  +  E[v2]  •  xx 

or  substituting  for  the  expected  values , 

EtT2’  *  xs  +  Xipp  *2«s  +  EtTl1)  +Xip  Vxs  +  e[t2! 
However,  since: 

e[t2]  =  xs  +  e[t2] 


and  substituting: 


°p  *ip  X1  and  p  =  A.  x_ 
r  r  np  inp  / 


we  can  solve  for  E[t2]  as: 


e[t2]  -  pnp  xs  +  V  E'T1>  +  pp  xs  +  °p  Erx2] 


or: 


P  xs  _  pnp  E[T1] 
F[tJ  =  - - - ^ - - 


1  -  P, 


where: 

p  +  p  =  aAx.  +  (1  -  a)  A  x» 
P  np  1  2 

and: 

ctx^  +  (1  -  a)x2  =  x 


so  that 


P  = 


P  +  p 
P  np 


(5.136) 


(5.137) 


(5.138) 
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By  repeating  the  above  arguments  at  subsequent  nodes,  the  result  of 
Equation  (5.137)  can  be  seen  to  be  a  general  iterative  relationship 
such  that. 


P  X  +  p  E[T  ] 

Eft  )  =  - ng - — 

n  1  -  p 


(5.139) 


with: 


e[Ti] 


PPXs 

1  -  P. 


(5.140) 


This  relationship  can  be  expressed  in  a  closed  form*  as: 


X. 


E(t„] 


1  -  P 


-  (M 


which  for  pp  equal  to  zero,  becomes  the  FCFS  result  of  Equation 
(5.117). 

If  we  oonpare  the  two  results,  we  see  that  unlike  the  FCFS 
system,  the  priority  system  has  a  non-zero  expected  gap  at  the  first 
node,  but  that  both  have  the  same  limiting  value  as  n  beocres  very 
large.  The  latter  effect  is  not  obvious,  but  can  be  shown  by  noting 
that: 


p  =  p  +  p 

P  np 


such  that: 


P  =  p  -  p 
np  M  Mp 


This  form  is  a  variation  of  the  closed  form  representat  ion  developed 
by  Fultz  from  the  iterative  result  of  Equation  (5. 1  "*9). 
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which  is  less  than  the  denominator,  1  -  p^,  and  therefore  in  the  limit, 


lim(i -=v) =0 

IT**3  \  Mp/ 


leaving : 


lim  E[t  I  priority  system]  =  -r— 2 —  x 
n-Kx>  n  1  -  p  s 


which  is  identical  to  the  limit  for  the  FCFS  case 

These  effects  are  shown  in  Figure  5.2.5  for  various  values  of 

p,  and  with  p  and  p  being  arbitrarily  made  equal.  This  assumes 
p  np  ~ 

that  ax  is  equal  to  (1  -  a)xnp  in  Equation  (5.138),  which  from  the 
priority  threshold  analysis,  seems  reasonable. 

The  preceding  analysis  was  based  on  the  assumption  that  all 
of  the  packets  of  a  message  arrive  at  the  source  IMP  simultaneously, 
i.e.,  as  a  bulk  arrival.  If  we  assign  a  finite  rate  to  these  HOST-to- 
IMP  transfers,  such  that  a  full  packet  requires  seconds  to  crass 
this  boundary,  then  the  analysis  must  be  modified  to  include  this 
effect.  The  interpacket  gap  at  the  output  of  the  originating  node  then 
becomes: 


E[Tli  =  ‘inp  *2  X1  +  Xip  VXs  +E[,in 


(5.141) 


which  is  a  modification  to  Equation  (5.131)  to  include  the  non-priority 
interference.  Solving  for  the  average  gap  time,  we  obtain: 


E[t1)  = 


p  X.  +  p  X 
np  1  p  s 


(5.142) 


The  delays  at  subsequent  nodes  can  be  determined  by  the  previous 
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>  p  -  0.8 


p  *  0.6 


pm  0.4 
p*  0.2 


Figure  5.2.5.  The  Expected  Interpacket  Gap  for  the  Priori  ly  System  After  Going  Through  n  Nodes. 


iterative  relationship, 


°  xs  +  pnp  ElTn-l1 


ElTn1  "  l-o. 


which  leads  to  the  closed  form  result: 
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(5.143) 


E(Tn]  = 


where  Xg  and  are  the  full  packet  service  times  for  the  IMP- IMP 

and  HOST- IMP  transmissions  respectively,  then  xn  is  the  interpackot 
gap  at  the  output  of  the  nw  s tore -and- forward  node.  (This  result 
also  shows  that  the  non- zero  HOST- IMP  transfer  time  deos  not  effect  the 
limiting  value  as  n  becomes  arbitrarily  large.) 

5.2.3  Experimental  Verification  of  the  Packet  Separation  Model 

A  conplete  experimental  verification  of  die  packet  separation 
model  would  require  the  ability  to  generate  artificial  traffic  at  each 
node  along  the  s tore-and- forward  path  of  a  test  message.  Since  such 
facilities  are  available  at  only  one  site,  the  UCLA  Network  Measurement 
Center,  the  verification  test  was  much  less  extensive  than  would  normal¬ 
ly  be  desired. 

The  test  was  designed  to  test  the  validity  of  Equation  (5.143) 
for  the  special  case  of  n  =  1, 


EfrJ  = 


(5.144) 


The  interference  traffic  was  generated  by  the  pseudo- random  artificial 
traffic  generator  (as  described  in  Section  2.5.8)  with  the  traffic  in¬ 
tensity,  o,  being  determined  by  selection  of  the  average  message  length. 
The  number  of  active  message  generators  was  a  compromise  between,  (1) 
the  desire  to  avoid  alternate  routing,  and  (2)  the  desire  to  minimize 
the  RFTffl  dependency  of  the  arrivals .  Eight  generators  were  found  to 
meet  both  desires  satisfactorily. 
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The  generation  of  the  multi-packet  test  traffic  was  originally 
planned  utilizing  three  different  techniques  bo  obtain  a  range  of  val¬ 
ues  of  the  parameter,  X^  (the  time  required  for  the  serial  "arrival"  of 
a  packet) .  The  "fake  HOST"  pseudo-message  generator  facility  within  the 
IMP  was  selected  to  provide  a  minimal  value  of  X^  of  3.3  msec.,  which 
is  the  typical  time  required  for  the  background  program  to  generate  a 
full  packet  of  data.  Similarly,  the  "fake  HDST"  generated  traffic  at  a 
neighboring  IMP  was  utilized  for  the  maximal  value  of  X^  =  23.0  msec, 
which  is  the  time  required  for  the  sericl  arrival  of  a  full  packet  at 
the  UCLA  IMP,  which  in  that  case  serves  as  a  store- and- forward  node. 

The  third  source  of  multi-packet  traffic  was  bo  be  from  the  network 
measurement  center  itself  since  the  interference  traffic  and  the  test 
traffic  could  be  created  using  separate  message  generators  and  message 
leaders  (to  trace  only  the  test  traffic).  However,  the  latter  condition 
does  not  provide  a  meaningful  set  of  interference  traffic  since  both 
the  test  traffic  and  the  interference  traffic  must  arrive  on  the  same 
serial  HOST- IMP  interface,  so  only  the  theoretical  curve  is  shown  for 
the  case  of  X^  equal  to  10.5  msec. 

The  theoretical  values  of  the  inter-packet  gap  time,  t,  for 
various  message  lengths  and  values  of  X ^  are  plotted  in  Figure  5.2.6 
along  with  the  test  data  frcm  the  experiment.  The  increase  in  the  value 
of  x  for  very  short  messages  is  due  to  the  fact  that  mast  of  these 
messages  are  of  the  priority  class,  and  therefore  cause  an  increased 
amoun'-  of  interference.  Equation  (5.144)  indicates  that  for  the  special 
case  in  which  all  of  the  interference  traffic  is  of  the  priority  class, 
the  value  of  x  will  not  be  a  function  of  X^,  and  this  effect  was 
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Figure  5.2.6.  A  Comparison  of  Measured  and  Theoretical  Inter-Packet 
Gap  Times  Due  to  Interference  Traffic. 


also  verified  by  the  experimental  data.  This  effect  is  due  to  the  fact 
that  the  only  arrivals  that  will  contribute  to  the  inter-packet  gap  are 
those  that  occur  during  the  Xg  seconds  that  a  packet  requires  for 
service,  and  the  subsequent  arrivals  during  their  service,  etc.,  as 
expressed  in  the  iterative  result  of  Equation  (5.131). 
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The  experimental  and  theoretical  results  agree  reasonably 
well  for  both  the  case  of  X1  equal  to  3.3  msec,  ard  for  X1  equal  to 
23.0.  Both  the  shape  of  the  curves  and  the  magnitude  of  the  inter¬ 
packet  gap  (as  measured  by  trace  data  at  the  destination)  indicate 
that  the  model  represents  the  actual  system  behavior  satisfactorily. 

The  theoretical  curves  include  the  effects  of  the  acknowledge¬ 
ments,  the  priority  messsages ,  and  the  non-priority  messages  by  con¬ 
sidering  the  expected  service  time  to  be: 

E[x]  =  E[x  |  an  ACK]  •  Prtan  ACK] 

+  E[x  |  a  pri.  mes.]  •  Pr[a  pri.  mes.] 

+  E[x  |  non-pri.  mes.]  *Pr[non-pri.  mes.] 

The  acknowledgements  are  in  response  to  the  HFNM's  (Request  For  Next 
ftessage) ,  and  therefore  occur  with  the  same  frequency  as  messages ,  such 
that: 

Prtan  ACK]  =  0.5 

Messages  are  considered  to  be  in  the  priority  class  if  they  are  less 
than  or  equal  to  10  bytes  in  length,  which  for  the  exponential  density 
of  iressage  length,  results  in: 

-10/j 

<\n=  1'e 

where  £  is  the  average  messaqe  length  in  bytes,  and  is  the 
fraction  of  messages  that  are  in  the  priority  class.  (The  ACK's  will 
also  be  in  the  priority  class) .  The  expected  service  time  for  a  prior¬ 
ity  message  can  be  found  from  the  truncated  exponential  result  of 
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Appendix  A, 


*p  =  El\  I  i  V 


and  the  expected  service  time  for  a  non-priority  message  is  also  shown 
to  be: 


Xnp  =  E[j^  |  >  Hp]  =  Hp  +  1/u 

Using  these  results,  one  can  write  the  expected  service  time  (of  mes¬ 
sages  and  ACK's)  as: 

E[x]  =  0.5Xa  +  0.5o^[XQ  +  xp]  +  0.5(1  -  o^) (Xp  +  XT  +  1/u] 

where  the  first  two  terms  represent  the  priority  service  times,  and  the 
last  term  considers  the  non-priority  service  times.  Hie  equation  is  of 
the  form: 

E[x]  =  a  E[x  |  priority]  +  (1  -  a)E[x  |  non-priority] 

and  if  multiplied  by  the  net  arrival  rate  (of  messages  and  ACK's) , 
produces : 

\  E[x]  =  A{0.5Xa  +  0.5om[X0  +  xp]}  +  1(0.5 (1  -  o^) [XQ  +  ^  +  1  /y]J 
where: 

X  =  X  +  X  =21 
am  m 

with  the  subscripts  referring  to  ACK's  and  messages.  The  values  of  the 
priority  and  non-priority  interference  traffic  intensities  can  then  be 
written  as: 
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pp  -  2y°-5Xa  +  °-5%lJ!0  +  V1 

and: 

"np  =  2Xm(0-5(1  -  V[xo  +  *r  +  Vnl> 
with  the  ccnposite  traffic  intensity  being: 

p  =  p  +  p 

If  we  consider  an  exanple  in  which  the  messages  have  an  average  length 
of  20  bytes,  we  find  that  40%  of  the  messages  are  in  the  priority  class, 
and  such  messages  have  an  average  service  time  of  3.44  msec.,  compared 
to  an  average  of  7.5  msec,  for  the  non-priority  messages  (including  the 
overhead  characters  for  each) .  The  service  time  of  an  acknowledgement 
is  3.0  msec. ,  and  when  combined  with  the  priority  messages,  produces  a 
traffic  intensity  of: 

p„  =  A  [3.0  +  0.4(3.44)]  =  0.175 
p  m 

where  A^  is  1/(25  msec.)  for  the  artificial  traffic  being  generated. 
The  traffic  intensity  for  the  non-priority  traffic  can  be  found  in  a 
similar  manner,  since: 

»np  -  V(1  '  “m><X0  +  *r  +  1/un 

which  becomes: 

pnp  =  %!  [2*9  +  1*6  +  3-21  =  °-185 

such  that: 


pc  = 


P  + 
P 


np 


0.36 
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The  expected  value  of  t  can  then  be  calculated  for  the  kncwn  value  of 
X  equal  to  23.0  msec.,  and  for  the  desired  value  of  X^.  These  re¬ 
sults  are  tabulated  in  Table  5.2.1  along  with  the  associated  values  of 


P 


and  p  . 
np 


TABLE  5.2.1 

THEORETICAL  VALUES  OF  THE  INTER-PACKET  GAP,  T 


Avg.Mes. 

Length 

(bytes) 

P 

% 

PP 

PnP 

Expected  Value  of  x 

X.  =3 . 3 

1 

X^IO.5 

X1=23.0 

0 

.23 

1.00 

.23 

.00 

6.9 

6.9 

5.9 

8 

.28 

.72 

.22 

.06 

6.7 

7.3 

8.3 

20 

.36 

.39 

.18 

.18 

5.8 

7.3 

10.1 

!  40 

.49 

.22 

.15 

.34 

5.3 

8.3 

13.3 

|  60 

.63 

.15 

.14 

.49 

5.6 

9.7 

16.8 

80 

.77 

.12 

.14 

.63 

6.0 

11.3 

20.4 

5.2.4  The  Special  Case  of  Exponential  Message  Lengths 

As  in  so  much  analysis,  the  assunption  of  an  exponential 
service  requirement  simplifies  the  analysis  by  decoupling  several  ef¬ 
fects  which  would  otherwise  be  highly  interdependent.  This  attribute 
of  *"he  exponential  density  is  the  well  kncwn  memory  less  property,  i.e., 
the  conditional  density  for  the  case  of  x  >  X  is  also  exponential 
with  the  same  parameter  value.  This  aspect  of  the  exponential  p.d.f. 
is  useful  in  the  segmented  message  case  because  of  the  fact  that  cut¬ 
ting  off  an  arbitrary  number  of  bits  from  the  beginning  of  a  message 
does  not  affect  the  statistics  on  the  remaining  portion  of  the  message. 
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We  will  take  advantage  of  this  memory  less  property  i;  the  following 
analysis,  and  subsequently  will  evaluate  the  results  by  an  actual 
network  experiment. 

We  will  assume  that  message  lengths  are  considered  to  be 
random  variables ,  but  that  messages  are  partitioned  into  packet  length 
segments  for  transmission  through  the  network.  That  is,  the  bits  of  a 
message  from  the  HOST  are  to  be  examined  serially,  until  £p  bits  have 
been  received,  at  which  time  a  packet  is  formed,  and  the  bit  count  is 
restarted.  The  service  time  for  such  a  packet  is  assumed  to  be  Xg 
seconds,  such  that  for  the  exponential  messaoe  length  density  with  a 
mean  message  service  requirement  of  1/p. 

‘yXs 

Pr [message  service  <  XsJ  =  1  -  e  ‘  (5.145) 


and  due  to  the  memory  less  property ,  this  same  probability  applies  to 
the  remaining  service  requirement  after  an  arbitrary  nunber  of  packets 
have  already  been  formed  from  the  message.  Therefore ,  since  the  proba¬ 
bility  that  the  message  service  will  not  be  ocnpleted  at  any  given 
packet,  is: 


_uXs 

Pr (service  not  completed]  =  e 


(5.146) 


Such  a  process,  in  which  the  probability  of  continuing  is  the  same  at 
each  epoch  in  time  until  a  "success"  is  finally  reached,  leads  to  a 
geometric  distribution  of  the  number  of  trails,  such  that  the  number  of 
packets  in  a  message  is  governed  by  the  probability  law: 


p(n) 


n  1,2, j, . . . 


(5.147) 
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and  the  expected  number  of  packets  per  message  is  simply: 

Klnl  =  ]/(l  -  o  ,Xf?)  ('*».  1 48) 

These  results  are  well  known,  and  have  been  utilized  extensively  in 
analysis,  including  as  a  model  for  the  number  of  time- sharing  quanta 
required  to  service  a  request  (AD69) .  In  many  ways,  the  problem  of 
multiplexing  messages  at  a  oormuni cations  channel  is  similar  to  that  of 
time-shared  systems,  and  many  of  the  following  results  may  be  applicable 
in  both  fields. 

If  nessages  are  not  segmented,  then  the  queueing  delay 

analysis  is  simply  that  of  the  well  known  fl/M/1  case,  i.e.,  messages 

arrive  at  an  average  rate  of  X  messages  per  second  and  require  an 

average  service  time  of  1/p  seconds  per  job.  The  server  utilization, 

p,  for  such  a  system  is  simply  X/p,  and  we  are  interested  in  comparing 

this  simple  case  with  the  segmented  message  situation.  The  following 

argument  considers  these  factors  for  the  two  cases,  and  shews  that  for 

the  M/M/1  system,  the  server  utilization  for  the  contiguous  message 

case,  p  ,  is  the  same  as  for  the  seqmrnted  message  case,  p  ,  in  which 
m  P 

the  message  is  partitioned  into  fixed  length  packets  with  a  service 
requirement  for  a  full  packet  of  Xg  seconds. 

We  know  that  for  the  contiguous  message  case,  the  arrival 
rate  and  average  service  time  are: 

X  =  X 
m 

and 

\m 1/v 


272 


such  that: 


pm  ’  V  \  "  x/u 

For  the  segmented  message  case,  the  arrival  rate  of  packets  is  more  than 
the  corresponding  rate  for  messages  by  a  factor  equal  to  the  average 
number  of  packets  per  message, 

Ap  =  A  •  E[no.  of  pkts.  per  mes.J 
which  from  Equation  (5.148)  is: 


The  expected  service  time  of  a  packet  can  be  found  from  the  servio* 

time  distribution  which  is  a  truncated  exponential  with  an  inpulse 

function  at  x  *  X  .  The  first  manent  of  such  a  function  is  shown  in 
s 

Appendix  A  to  be: 

*p  =  E[x  |  0  <  x  <  Xs]  =  i(l  -  e  S) 

such  that: 

X  1 1.  ■pXs\ 

°p  *  XP  XP - ^r-  •  ' e  I 

1  -  e 

or: 

pp  =  A/p  (5.150) 

which  is  precisely  the  same  as  the  result  for  pm  and  thereby  n roves 

that  the  two  values  of  p  are  equal. 

If  we  pursue  this  same  line  of  reasoning,  we  can  obtain  some 
additional  results  relating  the  contiguous  and  segmented  message 
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situations.  One  such  case,  which  does  not  necessarily  meet  with  one's 
intuitive  expectations,  relates  the  average  number  of  packets  in  the 
queue  for  a  segmented  system  to  the  number  of  messages  in  queue  for  a 
contiguous  system.  If  there  are  an  average  of  N  packets  per  message, 
the  intuitive  expectation  would  bo  to  find  more  than  one,  i.e.,  approxi¬ 
mately  N  packets  in  the  queue  of  the  segmented  system  for  each  mes¬ 
sage  in  the  queue  of  a  comparable  contiguous  message  system.  However, 
we  will  show  in  Theorem  5.2.3  that  just  the  opposite  is  true,  if  we  can 
assume  that  the  packet  arrivals  obey  a  Poisson  process.  This  is  not 
necessarily  an  acceptable  assumption  for  the  case  where  the  network 
traffic  is  low,  since  in  such  cases  the  packets  of  a  message  would  flow 
in  a  reasonably  contiguous  manner,  and  the  interarrival  times  would  be 
deterministic,  i.e.,  X  seconds  apart  (except  for  the  arrival  of  the 

b 

first  packet) .  At  higher  traffic  loads  the  packets  tend  to  separate  and 
would  have  more  random  interarrival  times,  which  is  encouraging  since 
the  analysis  is  of  more  concern  at  higher  network  loading.  A  second 
effect  which  introduces  a  randomization,  is  the  multiplicity  of  input 
channels.  Because  the  arrivals  of  interest  are  arrivals  at  the  output 
modem  queue,  and  a  given  packet  may  have  reached  the  IMP  frem  any  one  of 
several  input  channels,  the  composite  arrivals  to  the  queue  have  a  more 
random  pattern  than  the  arrivals  on  any  one  input  channel.  However,  the 
validity  of  the  Poisson  arrival  assumption  should  be  carefully  con¬ 
sidered  in  using  the  theorem  result. 

Theorem  5.2.3 

The  expected  number  of  packets  in  queue  for  a  segmented  mes¬ 
sage  case  is  less  than  or  equal  to  the  expected  number  of 
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messaaes  in  queue  for  a  corresponding  contiguous  message 
system,  for  the  case  of  Poisson  arrivals  and  exponential 
message  service  requirements. 

Proof • 

Hie  expected  nunber  of  messages  in  the  queue  for  a  contiguous  message 
M/H/l  system  is  known  to  be: 

2 

E[no.  of  mes.  in  Q  |  con  tig.  mes.]  =  ^  (5.151) 


The  expected  nunber  of  packets  in  the  queue  for  the  segmented  case  can 
be  shown  by  use  of  the  Pollaczek-Miinchine  formula  to  be: 


E[no.  of  pktn  in  Q  |  seg.  mes.l 


2  T 

.  V  *2 

~  rrr^ET 


(5.152) 


where  Ap  is  the  arrival  rate  of  packets,  pp  is  the  server  utiliza¬ 
tion  factor  for  packets  which  was  shown  previously  to  be  merely  equal 
to  p,  and  xp  is  the  second  moment  of  the  packet  service  time.  We 
know  from  Equation  (5.149)  that  the  arrival  rate  of  packets  is- 


and  frcm  Appendix  A,  the  second  moment  of  the  service  time  is: 

^  -  -|(l  -  a  +  ux  >e '“*) 

such  that  the  expected  number  of  packets  in  the  queue  becomes: 

-  /  x  f  M1  - a + ^s) 

■Vl  -MXJ  2(1 -p) 

'1  -  e  ' 
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or: 


nP  =  r^T 


1  -  (1  +  yxs)e 


-“v 


(5.153) 


where  we  have  replaced  A/y  by  its  equivalent,  p.  Sino_*  the  factor 
involving  p  is  equal  to  the  expected  nurrber  of  messages  in  queue  for 
a  contiguous  message  system,  we  must  then  shew  that  the  second  factor 
is  always  less  than  unity  to  prove  the  theorem.  This  can  be  done  by 
showing  that  the  numerator  is  never  greater  than  the  denominator  for 
all  0  <  X  ,  i.e.. 


(*■ 


-^s  v 
e  -  UXse 


]ir 


-yx. 


1  -  2e 


•  (."fl 


(5.154) 


Subtracting  the  first  two  terms  from  both  sides  results  in: 


Dividing  by  the  common  term  reverses  the  inequality  due  to  the  negative 
sign,  such  that: 


For  yXg  >  1,  the  inequality  is  obviously  true.  For  yXg  <  1  we  can 
expand  the  exponential  in  an  infinite  series,  such  that: 


O 

[yXj]  >  [l  -  (l  -  yxs  + 


(yxs)2 

~~Tl 


or: 
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? 

0  > 


<UXS)3 

31 


and  since  uX  <1,  the  absolute  value  of  each  term  decreases  and  so 
s 

the  first  term  is  dominant,  and  the  inequality  must  hold.  Therefore, 
the  average  nunter  of  packets  in  the  queue  as  expressed  in  Equation 
(5.153)  must  be  less  than,  or  at  most  equal  to,  that  of  the  contiguous 
message  case,  which  proves  the  theoran. 


Corollary  1 

The  expected  nutter  of  packets  in  the  queue  for  the  segmented 
ires  sage  case  of  Theorem  5.2.3  will  vary  from  that  of  the 
M/M/1  case  for  X  -*■  “,  to  that  of  the  deterministic  service, 
M/t)/l  case  for  Xg  0. 


Proof: 

The  expected  number  of  packets  in  the  queue  is  expressed  in  Equation 

(5.153)  for  a  full  packet  service  requirement  of  Xg  seconds.  Both 

the  numerator  and  the  denominator  cf  the  term  in  brackets  are  functions 

of  X  and  both  go  to  zero  as  X„  becomes  very  small.  The  use  of 
s  s 

L' Hospital's  rule  produces; 


lim 

UXs+0 


1  -  (1  +  yXg)e 


-yx„ 


(l  -  .**•)* 


=  lim 

“V* 


(1  +  uXg)e 


-yx, 


L  2 


(l  -  .^) 


-yx  \  -yXc 


or: 


lim 

•V0 
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which  is  still  indeterminate.  A  second  application  of  the  rule  results 


in: 


lim 

UX  >0 
s 


1 

2 


Therefore,  we  have  the  limiting  case  of: 


lixu 

^v° 


°p 


p T 


(5.155) 


which  is  the  expected  number  in  queue  for  an  M/D/1  queue t  ng  system. 
This  result  can  be  seen  intuitively  by  noting  that  as  Xg  Leccmes 
small,  most  of  the  packets  will  be  full  length  packets  and  will  there¬ 
fore  require  a  fixed  (deterministic)  service  time. 

At  the  other  extreme,  as  Xc  becomes  very  large,  the  system 
reverts  to  the  contiguous  message  M/to/1  case,  with: 


lim 

pXg-*00 


n  = 


7T 


2 

P _ 

-  p) 


(5.156) 


Thus  the  two  extreme  cases  are  shewn  to  be  the  M/D/1  system  for 

uX  -K)  and  the  M/tyl  system  for  pX  which  proves  the  corollary, 

s  “ 

Vfe  noted  in  Section  5.2.1  that  the  arrival  epochs  changed 
when  we  segmented  messages  because,  by  definition,  an  arrival  occurs 
when  the  last  bit  of  a  contiguous  stream  is  received.  Making  these 
message  segments  shorter  reduced  the  unfinished  work  in  the  system,  and 


from  the  corollary  result,  reduces  the  expected  unfinished  work  to  an 


arbitrarily  snail  value  since  we  have  a  lesser  nunber  of  message  seg¬ 
ments  in  the  queue,  and  each  segment  is  of  arbitrarily  small  size.  Of 


2n8 


course,  in  practice  one  must  include  sane  fixed  length  header  and 
catmunications  control  information  with  each  such  segment,  so  it  is  not 
practical  to  segment  messages  to  that  extent.  This  leads  one  to  assume 

that  there  must  be  sane  optimal  packet  length  which  will  be  a  function 

/ 

of  both  the  overhead  and  the  average  message  length,  and  such  an 
extension  of  this  analysis  is  considered  in  Section  5.2.5. 

5.2.5  The  Optimal  Segment  Size 

We  observed  in  Section  5.2.3  that  making  a  segment  (or  packet 
in  ARPA  network  terminology)  as  small  as  possible  improved  the  network 
performance,  and  therefore,  indicated  that  segmentation  should  be  done 
in  very  small  increments.  However,  each  packet  must  contain  identifica¬ 
tion  information  (its  header)  and  line  control  characters  which  con¬ 
stitute  a  fixed  overhead  per  packet  and  preclude  the  usage  of  very 
small  segment  sizes.  An  optimal  packet  size  must  then  exist,  which 
will  be  a  function  of  the  message  length  distribution  and  the  packet 
overhead. 

The  criteria  for  the  optimization  will  depend  on  the  aspect 
of  the  network  behavior  which  is  considered  to  be  most  sensitive  to 
this  selection,  and  might  include  the  preemptive- resume- like  behavior 
described  in  Section  5.1,  the  need  for  retransmission  due  to  errors  as 
considered  by  Kirlin  (KI69) ,  or  the  efficient  utilization  of  buffer 
storage.  This  latter  effect  is  felt  to  be  of  particular  significance 
in  the  ARPA  network,  and  will  be  considered  to  be  the  criteria  for  the 
optimization. 
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5.2. 5.1  The  Optimal  Segmentation  for  Exponential  Message  Lengths 

If  we  consider  an  exponential  density  of  message  lengths  as 
shown  in  Figure  5.2.7(a),  and  the  effect  of  segmenting  messages  into 
packets  of  length  L  ,  (e.g.,  by  taking  the  first  L  units  of  the 

message  and  forming  a  packet)  then  we  have  the  packet  length  density 
shown  in  Figure  5 . 2 . 7  (b) ,  which  includes  an  impulse  function  whose  area 
is  equal  to  that  of  the  portion  of  the  density  for  l  >  Ls«  The  re¬ 
maining  portion  of  the  density  then  becomes  the  new  density  of  interest, 
conditioned  on  the  fact  that  1  >  L,.  For  the  exponential  density,  the 
conditional  form  is  identical  to  the  original  function,  a  characteristic 
which  is  unique  to  the  exponential  and  leads  to  its  "manoryless"  prop¬ 
erty  which  makes  it  so  tractable  in  analysis. 

Another  aspect  of  the  optimization  is  the  selection  of  the 
buffering  attribute  to  be  optimized,  e.g.,  to  minimize  the  total  wasted 
space,  or  seme  other  criteria  of  concern.  The  selected  criterion  was 
to  optimize  the  efficiency  of  buffer  storage  utilization,  i.e. ,  to 
maximize  the  ratio  of  effective  storage  utilized  for  message  text  rela¬ 
tive  to  the  total  storage  required  for  the  fixed  length  block,  the 
header  information,  and  any  other  packet  overhead.  (The  line  control 
characters  do  not  need  to  be  considered  since  they  are  hardware  genera¬ 
ted,  and  we  are  only  concerned  with  manory  utilization.  However,  any 
necessary  pointers  for  linking  buffers  should  be  included.) 

If  we  assume  an  exponential  distribution  of  message  lengths 

with  a  mean  length  of  1  characters ,  a  maximum  text  length  of  L 

m  a 

characters  per  buffer,  aid  a  header  of  H  characters,  we  will  have  an 
average  buffer  efficiency  of: 
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0 

(a)  THE  ORIGINAL  EXPONENTIAL  DENSITY 


(b)  THE  TRUNCATED  DENSITY 


Q 


(c)  THE  CONDITIONAL  DENSITY 


Figure  5.2.7.  Tha  Effect  of  Segmentation  on  the  Exponentential  Density  of  Message  Lengths. 
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(5.157) 


8,  trr 

n  *  L  +  H  *  L  +  H 
s  s 

whore  I  is  the  moan  of  a  truncated  exponential  as  derived  in  Appendix 

A.  This  efficiency  is  shown  in  Figure  5.2.8  as  a  function  ot  length 

L  for  several  values  of  H.  Both  L  and  H  are  shewn  normalized 

s  s 

with  respect  to  the  mean  message  length,  l  . 

The  optimal  value  of  is  the  point  at  which  the 

efficiency  is  maximized.  We  can  therefore  calculate  these  optimal 
points  by  differentiating  Equation  (5.157)  with  respect  to  Lg,  which 
yields : 


dn 

dl^ 


■LA 

s  m 


e 

L  +  H 
s 


v1  - 


(Ls  +  H) 


.2 


=  0 


such  that: 


-L  /l 
s'  m 


[L  +  H]  - 


l[l  - 
m 


-L  /l 
s  m 


]  5  0 


(5.158) 


A  root  of  the  equation  will  be  the  value  of  the  length,  Lg,  at  which 
the  optimal  efficiency  occurs.  If  we  find  the  proper  root  of  this 
equation  and  then  insert  it  in  Equation  (5.157) ,  we  can  find  the  equa¬ 
tion  for  the  locus  of  the  optimal  points  (shewn  as  a  dashed  line  in 
Figure  5.2.8).  However,  to  avoid  the  necessity  of  solving  Equation 
(5.158)  for  its  roots,  we  will  take  advantage  of  the  form  of  the  num¬ 
erator  of  Equation  ('.157)  and  note  that: 


U1  -  e'^S 


m 


Ls0  +  H 


=  e 


-Lst/V 
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1 

NORMALIZED  PACKET  LENGTH,  Ls/  Cm 

Figure  5.2.8.  Buffer  Utilization  Efficiency  at  a  Function  of  the  Normalized  Packet 
Length  and  Packet  Storage  Overhead. 
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with  the  subscripts  indicating  that  LgQ  is  the  length  corresponding 
to  an  optimal  value  of  the  efficiency.  If  we  substitute  into  Equation 
(5.157)  we  obtain: 


n(opt)  =  e 


-L  JZ 

sO'  m 


(5.159) 


which  defines  the  locus  of  points  shown  in  Figure  5.2.8. 

The  optimal  buffer  size  as  determined  in  the  above  analysis 
was  later  found  to  be  consistent  with  an  earlier  result  obtained  by 
Vtolman  (WD65) ,  although  the  two  approaches  were  quite  different.  Wblman 
obtained  a  relationship  between  the  parameters ,  which,  using  our 
symbols,  is: 


L  s  /2~H  Z 
s  Y  m 

and  he  found  this  result  to  be  applicable  to  most  "smooth"  density 
functions,  i.e.,  those  which  do  not  have  large  discrete  or  concentrated 
probability  components .  His  approach  was  to  minimize  the  wasted  space, 
as  compared  to  our  maximization  of  the  utilized  space,  and  both  analyses 
considered  the  same  overhead  components.  His  result  has  the  advantages 
of  some  generality  and  of  being  an  explicit  equation  for  the  segment 
size  in  terms  of  the  average  message  length  and  the  overhead  per  packet. 
However ,  it  does  not  indicate  the  effect  of  non-optima 1  segment  size 
selection,  i.e.,  the  sensitivity  of  the  efficiency  to  the  choice  of  the 
packet  length  as  indicated  in  our  Figure  5.2.8.  The  two  results  are, 
therefore,  felt  to  be  very  ocrplementary ,  and  tend  to  confirm  the  re¬ 
sulting  selection  criteria  for  packet  size. 
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5. 2. 5. 2  Experimental  Verification  of  the  Model 


The  buffer  utilization  efficiency  can  be  viewed  as  the  ratio 
of  the  average  packet  size  (of  useful  information)  to  the  total  storage 
requirement  of  a  fixed  length  buffer.  The  latter  figure  includes  any 
packet  header  information,  marking,  padding,  and  the  necessary 
pointer (s)  for  linking  the  buffers  on  the  queues.  The  model  was  de¬ 
veloped  to  give  seme  indication  of  the  optimal  buffer  size  for  a  given 
average  message  length  (assuring  an  exponential  density  of  lengths) . 

A  logical  verification  test  would  be  to  vary  the  buffer  size  and  meas¬ 
ure  the  resulting  storage  efficiency,  n.  However,  it  is  difficult  to 
change  the  buffer  size  since  it  is  an  IMP  design  parameter,  so  an 
alternative  method  of  varying  the  average  message  size  was  pursued. 
Changes  in  the  parameter,  i-m,  are  equally  effective  in  changing  the 
ratio  L s/^m  an  variations  in  Lg,  and  result  in  the  efficiency  curve 
shown  in  Figure  5.2.9.  This  plot  was  obtained  by  generating  artificial 
traffic  with  an  exponential  distribution  of  lengths,  and  measuring  the 
average  information  content  of  packets  as  the  average  message  length 
was  varied.  Each  buffer  was  assured  to  consist  of  a  total  of  138  bytes 
of  message  content  plus  12  bytes  of  overhead) ,  with  a  resulting  effi¬ 
ciency  of: 


(#  bytes  sent/#  pkts.  sent) 
138 


(5.160) 


The  two  measured  values  were  the  number  of  message  content  bytes  trans¬ 
mitted  on  the  modem  channel,  and  the  number  of  packets  sent  on  that 
sane  channel.  (Both  of  these  values  are  available  frem  the  accumulated 
statistics  as  described  in  Section  2.5.1) 
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BUFFER  UTILIZATION  EFFICIENCY  , 


c- 


Figure  5.2.9.  Buffer  Utilization  Efficiency  as  a  Function 

of  Average  Message  length  (Exponential  Density) 
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Figure  5.2.S  differs  from  the  theoretical  curve  of  Figure 
5.2.8  since  the  latter  was  based  on  the  independent  variable  being  Lg. 
However,  a  second  theoretical  curve  is  shown  in  Figure  5.2.9  as  a 
function  of  i.^,  and  a  reasonably  close  agreement  can  be  seen  between 
the  theoretical  and  measured  data.  Hie  difference  at  higher  values  of 
is  due  to  the  fact  that  message  sizes  are  limited  to  a  maximum  of 
1000  bytes,  and  this  was  not  taken  into  account  in  the  theoretical 
model. 

This  same  data  can  be  compared  with  the  theoretical  curve  of 
Figure  5.2.8  by  rescaling  the  effective  overhead,  H,  to  be: 


H(eff.)  _  6 
~T  63 


(5.161) 


That  is,  the  value  of  H  is  six  for  L  equal  to  63  and  should  retain 

S 

that  same  nonralized  value  as  l  is  varied.  Therefore, 

m 

H(eff .)  -  6  £^63  (5.162) 

will  retain  the  effective  value  of  H  equal  to  approximately  0.1  as 

the  ratio  Lg/£.m  is  varied.  The  resulting  theoretical  and  measured 

values  are  compared  in  Figure  5.2.10,  and  again  indicate  a  reasonably 

good  match  between  theory  and  experiment.  The  same  truncation  effects 

that  influenced  the  upper  portion  of  the  curve  in  Figure  5.2.9  are  new 

seen  in  the  lower  values  of  L/l  ,  and  cause  the  two  curves  to  be 

s  m 

somewhat  different  in  that  range. 
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Figure  5.2.10.  Buffer  Utilization  Efficiency  at  a  Function  of  the  Normalized  Packet  Length 
(Bated  on  Ad  jut  ted  Valuet  of  the  Overhead,  H  ) 
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CHAPTER  6 


CCNCLUSIONS  AND  SUGGESTED  FUTURE  RESEARCH 

This  research  has  shewn  that  an  extensive  measurement 
capability  can  be  developed  within  a  store- and-forward  oatmunications 
network,  and  that  with  proper  care,  these  measuranent  facilities  can 
utilize  the  processing  and  transmission  facilities  of  the  data  network. 
However,  we  found  it  necessary  to  work  very  closely  with  the  network 
implementation  group  to  ensure  that  the  necessary  data  could  be  obtained 
with  a  minimum  perturbation  to  the  system  behavior.  These  measurement 
considerations  included  the  selection  of  the  proper  set  of  measurement 
facilities,  and  the  introduction  of  the  capability  to  selectively  en¬ 
able  or  disable  the  various  measurement  routines  at  each  node  in  the 
network. 

The  measurement  facilities  were  initially  utilized  to  verify 
the  design  and  operation  of  the  network,  and  as  part  of  this  activity, 
an  extensive  artificial  traffic  generation  capability  was  added  to  the 
set  of  measurement  tools.  This  artificial  traffic  generation  capability 
was  later  found  to  be  extremely  valuable  for  the  simulation  of  traffic 
flow  conditions,  and  was  subsequently  modified  to  include  random  mes¬ 
sage  length  and  transmission  time  capabilities,  in  addition  to  the 
original  deterministic  traffic  generation  capabilities.  The  importance 
of  artificial  traffic  generation  was  not  fully  appreciated  initially, 
but  evolved  to  be  one  of  the  more  valuable  network  measurement  tools. 
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Subsequent  network  measurement  activities  have  demonstrated 
that  the  network  performance  can  be  evaluated,  and  that  anomalous 
behavior  can  be  discovered  and  information  in  be  gained  regarding  its 
cause.  Hie  complementary  nature  of  the  measurement  and  modelihg 
efforts  was  also  demonstrated,  with  the  experimental  results  being 
utilized  j.n  several  instances  to  inprove  or  correct  the  models.  The 
models  have,  in  turn,  been  utilized  as  the  basis  of  further  network 
experimentation,  with  the  modeling  and  measurement  functions  being 
found  to  be  mutually  constructive  in  an  iterative  scheme  of  model 
development  and  verification  testing. 

Several  application  areas  are  envisioned  for  future  network 
experimentation  including  the  measurement  of  the  network  traffic  for 
use  in  determining  future  topological  and  bandwidth  changes ,  as  well  as 
to  identify  the  different  user  transmission  characteristics  such  as 
those  due  to  interactive  teletypewriter  activity  and  graphic  display 
data  requirements.  Measurements  cure  also  planned  to  determine  areas  in 
which  dynamic  control  mechanisms  oould  be  effectively  utilized  to  en¬ 
hance  the  network  performance ,  and  to  determine  more  extensive  measures 
of  such  system  performance. 

A  number  of  possible  measurements  of  interest  would  require 
the  cooperation  of  one  or  more  HOST  facilities  to  obtain  the  necessary 
data.  These  experiments  include  measurements  of  the  performance  of  the 
overall  network  facilities  including  the  HDST- implemented  protocol  func¬ 
tions,  and  the  degree  of  system  degradation  imposed  by  such  protocol 
imp  lementa tions  in  terms  of  both  the  memory  space  and  response  time 
considerations.  A  final  suggestion  for  future  research  which  may  be  of 
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particular  interest  in  determining  the  viability  of  network  tedmiques, 
is  to  determine  the  degree  to  which  users  at  the  various  sites  are  able 
to  utilize  each  other's  resources  rather  than  regenerating  a  local  ver¬ 
sion  of  the  same  facilities.  Such  measurements  would  also  determine 
the  features  that  should  be  available  in  future  program  and  hardware 
developments  that  will  nake  them  more  useful  in  a  network  environment. 
It  is  hoped  that  such  future  measurement  activities  can  extend  and 
build  upon  the  measurement  facilities  which  were  developed  as  part  of 
this  research. 
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APPENDIX  A 


THE  MCMENTS  OF  THE  TRUNCATED  EXPONENT r\L  DENSITY 

The  analysis  of  both  the  two  class  priority  queueing  systan 
and  the  segmented  message  service  case  required  that  one  calculate  the 
moments  of  truncated  density  functions.  Such  moments  have  been  derived 
by  Adiri  (AD69)  and  also  by  Coffman  and  Kleinrock  (0068) ,  and  are  sun- 
mar  i  zed  in  the  following  equations,  although  the  notation  and  form  of 
the  results  have  been  modified  to  suit  our  purposes  more  directly 

a.  The  Truncated  Exponential  With  a  Deterministic  Ccrrponent 

Wien  messages  are  segnented  into  fixed  length  segments  or 
packets,  some  fraction  of  the  segments  are  of  the  maximal  length,  and 
result  in  a  deterministic  ooqpcnent  of  the  service  time  density  func¬ 
tion  as  shown  in  Figure  A.  1(b) .  The  density  also  has  a  fixed  offset  at 
the  origin  to  account  for  the  overhead  service  requirement  six*  as  the 
line  control  characters  and  the  header. 

The  moments  of  this  function  can  be  shown  to  be: 


1  ’^s 

E[x]  =  Xq  +  i  [1  -  e  S] 

and: 

E[x2)  -  X2  +  [1  -  (1  +  uXg)e  ^ ] 


(A.l) 


2*0  "yXs 
+  ~u  I1  “  e  3] 


Preceding  page  blank 


(A. 2) 
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*0 


(a)  THE  EXPONENTIAL  DENSITY  FUNCTION 


(b)  TRUNCATED  EXPONENTIAL  WITH  A  (c)  TRUNCATED  AND  RENORMALIZED 

DETERMINISTIC  COMPONENT  DENSITY 


(d)  A  RENORMALIZED  EXPONENTIAL  TAIL  FROM  THE  TRUNCATION 


Fifura  A.1.  Tha  Exponantial  Dansity  and  Savaral  Variations  of  its  Truncatad  Form. 
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b.  The  Truncated  and  Renormalized  Densi 


Hie  truncation  effect  in  the  priority  analysis  did  not  involve 
an  impulse  function  at  the  truncation  value,  and  therefore  resulted  in 
the  two  conditional  densities  shown  in  Figure  A.  1  (c)  and  (d) .  To 
differentiate  between  the  two  types  of  truncation,  and  to  be  consistent 
with  the  notation  used  in  the  priority  analysis,  we  shall  consider  this 
truncation  value  to  be  Xq  +  Xp*  The  moments  of  the  first  of  these  two 
truncated  functions  are: 


"V*T1 

.  l  e 

E[x  |  x  <  xQ  +  jy  =  xQ  +  i  |i  -  -2—^ 


(A.  3) 


E[x^  |  x  <  XQ  +  y 


xo  +  7  1‘ 


f  (uXJ  \  -ux_ 

K  +  -r-)e 


1  -  e 


r 

2Xq  U^e 

V  “UXp 

1  -  e  1 


(A. 4) 


Although  Adiri  did  not  consider  the  second  function  directly,  it  car.  be 
considered  to  be  a  special  case  of  part  (a)  with  Xq  becoming  equal  to 
X^,  and  xg  becoming  arbitrarily  large.  The  moments  are  then: 

E[x  |x  >  XJ.J  -  “  (A.5) 


E[x2|x  >XfJ  -  + 
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or: 
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APPENDIX  B 


THE  TRUNCATED  l/x  DENSITY  FUNCTION 


An  interesting  density  function  for  much  of  our  analysis  is 
the  truncated  density. 


f  (x)  »  l/x  In  M 


1  <  X  <  M 


(B.l) 


The  function  has  a  very  sharp  peak  at  its  initial  value,  x  *  1,  and  a 
relatively  flat  slope  near  its  maximum  of  x  *  M. 

The  moments  of  the  function  can  be  found  by: 

/*M 

Xn-  /  n-rcTT  dx 


which  can  be  integrated  for  the  general  case,  resulting  in: 


7-  * 


n  In  M 


M 


M°-  1 
n  In  M 


(B.2) 


Using  the  first  and  second  moments,  we  can  find  an  expression  for  the 
variance. 


2  (M2  -  l)ln  M  -  2(M  -  l)2 

j  - - 5 - 

2  (In  M; 


and  for  the  coefficient  of  variation , 


(B.3) 


c  = 


(M  +  l)ln  M 

TW  -  T r 


-  1 


1/2 


(B.4) 
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Itie  truncation  of  the  function  at  some  finite  upper  bound  is  necessary 
to  meet  the  init  area  requirement.  Also,  since  the  cumulative  distribu¬ 
tion  function  is: 

F(x)  ■  In  y/hi  M  (b.5) 

the  lower  bound  can  not  be  less  than  unity  (to  avoid  negative  values) . 
For  our  purposes,  this  lower  bound  shall  always  be  considered  to  be 
unity. 

The  1/x  density  has  several  valuable  features  which  make  it 
of  interest,  including  the  fact  that  it  transforms  into  a  uniform  den¬ 
sity  for  the  transformation  of  variable, 

K  =  logbx 

which  is  the  continuous  equivalent  of  the  log  histogram  reshaping 
effect.  The  availability  of  readily  calculated  higher  moments  is  also 
a  convenience  for  use  as  a  test  case  in  the  moment  transformation  work 
of  Section  3. 3. 2.1. 
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APPENDIX  C 


TEST  CASES  FOR  TOE  MOMENT  TRANSFORMATIONS 


Pairs  of  f(x)  and  <f>(0  functions  are  needed  for  evaluation 
of  the  rrcment  transformations  derived  in  Section  3. 3. 2.1.  Since  these 
equations  require  the  calculation  of  many  higher  moments  of  <t>  (£) ,  we 
will  investigate  functions  with  readily  obtainable  moments,  and  in 
particular,  will  consider  uniform  and  exponential  forms  of  the  function, 

a.  Uniform  $(5)  Density 

The  logarithmic  transform  function  pair  for  which  <t>  (O  is 
uniform,  has  an  f  (x)  function  of  the  form: 

f  (x)  -  1/x  In  M  1  <  x  <  M  (C.l) 

and  fran  the  transformation  of  variables, 

<t>(K)  *  In  2/ln  M  0  <  C  <  log^.  (C.2) 

The  moments  of  the  function,  <P  (0  can  be  shown  to  be: 


rn  -  IiLgln/ 

5  -  LETT!  /  (n  +  .1) 

with  the  central  moments  being: 

0  for  n  =  odd 

Un  TlnMln/ 

[2  In  2J  /  (n  +  1)  for  n  =  even 


(C.3) 


(C.4) 
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Fran  Equation  (3.44) ,  we  have  the  relationship: 


x  =  1  +  C  In  2  + 


E 

21 


(In  2V  + 


£ 

n! 


(In  2) 


n 


(C.5) 


which  upon  substitution  for  the  moments  of  $(£)  becomes : 


1  + 


In  M 


(In  M)  t 

+  3(51)  + 


+ 


(In  M)n 

Tn'+'iy'fniT 


(C.6) 


A  iiore  convenient  form  can  be  obtained  by  adding  and  subtracting  unity, 
resulting  in: 


x  =  [(1  +  ln  M  +  +  (^f--  +  •••)  -  l]  (C. 7) 

The  expression  within  the  parentheses  can  be  recognized  as  the  infinite 
series  equivalent  of  the  exponential,  which  further  simplifies  to: 


The  resulting  expression  for  x  is: 

x  =  (M  -  l)/ln  M  (C.8) 

which  from  Equation  (B.2)  is  known  to  be  the  mean  value  of  the  1/x 
density.  This  result  serves  as  a  check  on  the  correctness  of  the  expan¬ 
sion,  but  does  not  answer  the  real  question  of  concern,  namely,  how 
rapidly  does  the  series  converge? 

We  could  pursue  this  problem  for  the  general  value  of  M, 
but  since  such  a  result  would  still  be  a  special  case  as  far  as  general 
p.d.f.'s  are  concerned,  we  shall  not  bother  with  such  a  pseudo-general¬ 
ity.  Instead,  we  shall  consider  the  special  case  shown  in  Figure  C.l 
for  which  the  mean  value  of  x  is  known  to  be  15.15.  Vtien  we 
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(a)  THE  f(x)  DENSITY  AND  ITS  LOGARITHMIC  TRANSFORMED  EQUIVALENT  DENSITY. 


(b)  THE  RELATIVE  CONVERGENCE  OF  THE  ESTIMATES  OF  x  FROM  THE 
MOMENTS  OF  #|). 


Figure  C.l.  The  Density  Functions  and  Convergence  of  the  Estimate  of 
the  Mean  for  the  Example  Considered. 
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calculate  the  moments  of  $(£)  and  substitute  these  moments  into 
Equations  (3.44)  arri  (3.50) r  we  find  that  the  former  converges  very 
slowly,  requiring  the  first  6  to  10  moments  to  obtain  a  reasonable  es¬ 
timate  of  the  nean,  while  the  latter  converges  much  more  rapidly .  In 
fact,  tte  expansion  in  terms  of  the  central  moments  converges  to  within 
0.3%  of  the  actual  value  when  only  three  moments  are  considered,  com¬ 
pared  to  a  1.0%  error  in  the  estimate  when  the  first  ten  moments  about 
the  origin  are  considered.  However,  these  figures  should  not  be  con¬ 
sidered  to  hold  with  any  generality,  or  as  necessarily  being  typical  of 
the  convergence  of  each  series. 


b.  Exponential  4>(£)  Density 


A  second  transform  pair  involves  a  uniform  <J>  (£)  density 
with  its  logarithmic  transform  function  being  an  exponential  in  the 
left  half  plane.  The  f(x)  function  is: 


f(x)  = 


0  <  x  <  1 
otherwise 


(C.9) 


and  the  corresponding  <J> (£)  function  can  be  shewn  to  be: 


<j>(U  =  In  2  e‘ 


£ln2 


<  C  <  0 


(C.10) 


This  extmple  is  of  particular  interest  because  it  does  not  result  in 
convergence  of  either  Equation  (3.44)  in  terms  of  moments  about  the 
origin,  or  Equation  (3.50)  in  terms  of  the  central  moments.  Hcwever, 
the  1  expansion  does  oscillate  in  a  reasonably  restricted  range 
about  the  mean  as  shown  in  Figure  C.2.  The  expansion  in  terms  of  mo¬ 
ments  about  the  origin  oscillates  between  zero  and  unity  because  of  the 
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ESTHETE  OF  THE  MEAN 


(a)  The  f  (x)  DENSITY  AND  ITS  LOGARITHMIC  TRANSFORMED  EQUIVALENT  DENSITY 


NO.  OF  MOMENTS  CONSIDERED 

(b)  THE  RELATIVE  CONVERGENCE  OF  THE  ESTIMATES  OF  x  FROM  THE  MOMENTS  OF  0(|j) 


Figure  C.2.  An  Example  in  Which  the  Moment  Transforms  Do  Not  Converge. 
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alternating  series  which  results  for  these  values  of  the  moments  of 
<J>  (£) .  The  functions  used  in  the  example  are  not  considered  to  be  rep¬ 
resentative  of  actual  densities  which  would  be  encountered  in  practice, 
but  are  a  convenient  functicn-pair  to  demonstrate  the  convergence 
problem. 
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APPENDIX  D 


MOMENT  CORRECTIONS  FOR  GROUPED  DATA 

The  calculation  of  moments  from  grouped  data  such  as 
histograms  can  result  in  systematic  bias  errors  depending  on  the  shape 
of  the  density  function  and  the  order  of  the  moment  being  estimated. 
Sheppard  (SH98)  developed  correction  formulas  for  gaussian-like  den¬ 
sities  using  the  Euler-^acLaurin  sum  formula  as  a  discrete  approximation 
to  the  definite  integral  representation  of  the  moments.  The  necessary 
and  sufficient  conditions  for  the  applicability  of  the  correction  for¬ 
mulas  are  met  by  functions  which  are  continuous  and  finite  over  the 
range  of  interest,  and  which  taper  to  zero  at  each  extreme.*  For  such 
functions,  the  first  moment  is  unbiased,  and  the  variance  has  a  cor¬ 
rection  of: 

(a2)c  =  (cr2)uc  -  h2/12  (D.l) 

where  the  subscripts  c  and  uc  denote  the  corrected  and  uncorrected 
estimates  respectively,  and  h  is  the  histogram  interval  size. 

Most  of  the  density  functions  that  we  have  considered  do  not 
meet  the  above  conditions,  since  the  exponential  and  other  "J-shaped" 
densities  do  not  taper  to  zero  snoothly  at  both  extremes.  Elderton 
(EX.33)  has  made  adjustments  for  these  shape  differences  using  an  em¬ 
pirically  derived  rule  based  on  observations  of  the  exponential  density, 

* 

A  more  formal  description  of  Sheppard's  corrections  can  be  found  in 
CramSr  (CR6i) . 
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out  in  a  companion  paper,  Peurson  shewed  that  these  corrections  degrade 
considerably  for  densities  with  shapes  that  radically  depart  fran  that 
of  the  exponential.  (One  such  deviation  is  the  so-called  "twisted 
J-shaped"  curve  which  has  a  large,  but  finite,  slope  at  its  origin.) 
Pearson  also  compared  Elderton's  corrections  with  other  adjustment  for¬ 
mulas  by  Pearse  and  by  Martin  which  were  more  accurate,  but  required 
considerably  more  computational  effort.  The  Pearse  method  also  included 
an  "abruptness"  coefficient  for  truncated  density  functions.  Abernathy 
(AB33)  took  a  different  approach  to  densities  which  do  not  meet  the 
restriction  of  Sheppard's  formulas,  in  that  he  raroved  the  need  for  the 
slowly  tapering  tails  of  the  density  at  each  extreme,  and  only  required 
the  existence  of  the  manents  of  the  density  function. 

These  manent  corrections  as  developed  by  Sheppard  and  as 
modified  by  Elderton,  Abernathy, and  others,  have  all  been  based  on 
uniform  interval  histograms.  We  will  take  a  different  approach  which 
is  not  limited  to  uniform  intervals,  and  which  takes  advantage  of  the 
information  available  in  the  neighboring  interval  heights.  For  example, 
if  we  considered  the  i^1  interval  by  itself,  we  could  only  assume  a 
uniform  distribution  across  the  interval.  However,  if  we  know  that  the 
interval  to  tb^  left  is  greater,  and  the  interval  to  the  right  is 
smaller  than  the  i^1  interval,  we  have  additional  information  regard¬ 
ing  the  most  likely  distribution  of  the  probability  mass  across  the 
til 

i  interval.  This  usage  of  neighboring  interval  heights  to  determine 
a  linear  slcpe  function  will  be  referred  to  as  a  first-order  approxima¬ 
tion  to  the  density  shape,  as  opposed  to  the  zero-order  approximation 
of  the  uniform  density  assumption.  This  terminology  and  approach  is 
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analogous  to  that  used  in  sanpled  data  reconstruction,  and  in  keeping 
with  the  analogy,  higher  order  approximations  are  of  rapidly  increasing 
carplexLty,  and  are  felt  to  have  marginal  value  considering  this  added 
complexity. 

In  order  to  develop  techniques  which  are  applicable  to  both 
uniform  and  log-histograms,  we  will  first  consider  a  more  general  ap¬ 
proach  to  histograms  as  shown  in  Figure  D.l.  The  intervals  are 


Figure  0.1.  A  First-Ordar  Approximation  to  a  Ganaraiizad  Histogram  Foimat. 

determined  by  a  functional  relationship  such  that  h(i)  is  the  width 
of  the  1th  interval.  If  out  of  a  total  of  N  observations  occur 
in  the  i^  interval,  then  the  probability  associated  with  tlx  interval 
is  rytt,  and  far  the  zero-order  approximation ,  the  elsnental  density 
function  is: 
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(D.2) 


^(x)  «  ni/Mh(i)  <  x  <  Xi+1 

Ihe  oarponent  of  the  manent  due  to  this  interval  for  any  approxi¬ 

mation,  f^(x) ,  will  be  denoted  by: 


AE[xn]i 


f^x)  dx 


(D.3) 


and  since  the  manent  contributions  are  additive,  the  mcment  estimates 
will  be: 


N-l 

Etx*]  =  £  AEIX0],  (D.4) 

i-0  1 


The  approximate  slope  for  the  1th  interval,  nu  ,  will  be  the  weighted 
average  of  the  slopes  to  the  midpoints  of  the  two  neighboring  intervals. 


ni 

ni-l  \ 

.  n}+1 

ni  ) 

Nirnr 

Nhfi  -  1)  ( 

+  (1  -  w/) 

tlh(i  + 

TT  "  SRTD  ( 

j[h(i 

-  1)  +  h(i)  ]  ) 

j[h(i) 

-  h(i  +  1)]  ( 

If  we  select  the  weighting  according  to  the  length  of  each  slope  seg¬ 
ment,  i.e. , 


h(i  -  1)  +  h(i) 

wi  “  h(i  -  1)  +Th(i)  +  h(i  +  TT 


(D.6) 


and: 


,  ..  _  h(i)  t  h(i  +  1) 

1  '  wi  -  m  -  ir+  2R7D“+"h7r+”Ty 

then  the  slope  becomes: 
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(D.7) 


,  [  ni+l _ ni-l 

3  2  LnH  (X  +  l)  ~  Nh(i  -  1)  . 

mi  h(i  -  l)  +  2h(i)  +  h(i  +  1) 


.th 


Hie  first  order  approximation  function  for  the  i  *  interval  is 


f i (x)  =  mix  +  Bi 


Xi  <  x  <  xi+l 


for  which: 


Ei(Xi  +  ^)“nVNh(i) 


such  that: 


Bi  ■  nkW  '  mi(xi  + 


Hie  function  can  then  be  written  as: 


£i(x)  -  V +  SK7IT  -  mi(xi ♦  ^rj 

The  ocnponent  of  the  n*"*1  manent  is  from  Equation  (D.  3) , 


AE[x 


Xi+1 

»  /  x^lm.x  +  B.] 


dx 


X. 

l 


or: 


«  n  x^2 

AE[x  ]i  =  mi  jppj 


x^1 

+  BiHTT 


xi+l 


which  becomes: 


(D.8) 


(D.9) 


(D.10) 


(D.ll) 


(D.12) 
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4*lx"Ii  -  AlK+i'"*2  -  ,xi)rrt2]  +  HTT[«i+i>ntl  -  oti»ntl]  »'13> 

\ 

where  nt,  B^,  and  are  as  defined  in  Equations  (D.7),  (D.10),  and 
Figure  D.l  respectively.  This  is  as  far  as  we  cam  carry  the  develop¬ 
ment  without  introducing  a  functional  relationship  between  the  h(i)'s 
and  the  X^'s  so  we  will  next  consider  the  two  special  cases  of  in¬ 
terest,  namely  the  uniform  and  log-histograms. 

a.  The  Uniform  Interval  Histogram 

For  an  M-interval  uniform  histogram,  all  of  the  intervals 
are  of  length  h  such  that: 


h(i;  =  h  i  =  0,1,2,. . .  ,M-1 

and: 

*  ih 


(D.14) 

03.15) 


with  the  range  of  x  equal  to: 

0  <  x  <  Mh 

If  we  evaluate  Equation  (D.12)  for  the  expected  value  of  x,  in  the 
i^  interval,  we  obtain: 


AEtxJi  «=  ^{[(i  +  1) h] 3  -  [ih]3}  +  ^{[(i  +  l)h]2  -  [ih]2| 


or: 


AE[x] 


mi^  f  2  )  Bi^  f  ) 

i  =  “3 — l3i  +  3i  ♦  4  +  -5— {2i  +  1( 


(D.16) 


Since  we  know  that  is  equal  to: 
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0. 


Bi  =  i£-mi[ih  +  2] 


then: 


iilxli  =  (3i2  +  3i  +  1)  ♦  -  m.  Oh  +  5>|(r)(2i  +  X) 


and  by  collecting  like  terms: 

m.h3  .2  r  1 

E[x]i  =  +  J-  (2i  +  1)  ^VNhJ 

which  upon  substituting  for  nt , 


(D. 17) 


AE[x].  =  (ih  +  |)  —  +  2lT  [n>l  '  ni-l]  (D*18 

becomes  the  desired  result.  The  first  term  is  the  conventional  can- 
ponent  of  the  mean,  and  the  second  term  is  the  first-order  correction 
component.  Note  that  if  is  greater  than  n^_^,  i.e.,  for  a 

function  with  a  positive  slope,  the  correction  is  positive  since  more 
of  the  mass  is  beyond  the  midpoint,  and  conversely,  if  the  slope  is 
negative,  the  mass  is  nearer  the  origin  so  the  correction  component  is 
negative.  The  total  correction  will  be  the  sun  of  these  components, 
but  we  must  also  account  for  the  end  conditions  where  the  slope  func¬ 
tions  are  undefined.*  A  reasonable  approach  was  found  to  be  to  assure 
that,  for  slope  considerations,  we  should  let  =  Og  and 
=  nM_1»  such  that  the  correction  formula, 


*1he  end  corriitions  correspond  to  the  "abruptness"  coefficient 
mentioned  earlier. 
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(D.19) 


correction 


M-l 

h  y 

24NiS 


(n 


i+1 


-  Vi> 


becomes: 


correction  =  ^  [n^  -  nQ]  (D.20] 

This  result  is  consistent  with  Sheppard's  correction  for  functions 
which  taper  to  zero  at  the  two  extremes  since  for  such  functions  the 
correction  would  be  zero. 

The  second  moment  correction  is  more  ocnplicated  because  of 
the  terms  being  raised  to  the  higher  powers,  but  can  otherwise  be 
evaluated  in  a  similar  manner.  However,  there  are  two  choices  for  the 
conventional  representation;  (1)  assorting  that  all  of  the  probability 
mass  is  located  at  the  center  of  the  interval,  at  ih  +  h/2,  and  (2) 
assuming  that  the  mass  is  spread  uniformly  over  the  interval ,  in  which 
case  the  effective  mass  location  would  be  ih  +  h//3.  After  consider¬ 
able  algebra,  one  can  show  that  the  oorponent  attributions  for  the 
first  case  cure: 


AEix'ii  -  [(ih  +  |)2  J£|  ♦  4  tnAh  *  <ih  +  $)  (ni+1  -  vill 


for  which  the  total  correction  becomes, 


-h 


correction  =  ^  %  ▼  t)^  +  n2  +  •••  +  rM_l) 


or  sinply, 


correction  * 


(D.21) 
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which  is  equivalent  to  Sheppard's  correction,  since  the  estimate  of 
variance  is: 

o2  =  E[x2]  -  {E[x]}2  (D.22) 

and  the  estimate  of  the  mean  was  unbiased  for  the  set  of  applicable 
density  functions.  The  combination  of  the  two  correction  formulas  of 
Equations  (D.20)  and  (D.21)  are  applicable  co  any  density  functions  for 
which  the  moment  integral  of  Equation  (D.3)  is  defined. 

For  the  second  case,  i.e.,  where  the  effective  probability 
mass  location  is  ih  +  h//3,  the  ccrporient  contributions  are: 

-  [(ih  +  h/A)2  ^i]  +  (ih  +  h/A>  j}g|[n1+1  -  vj 

for  which  the  total  correction  is: 


2  1 

correction  -  TBT  £  (i  +  ^)(Vl  ‘  Vl) 


and  results  in  a  first-order  correction  of; 


correction  = 


-h2/3 

“23“ 


(D.23) 


This  correction  is  of  the  same  sign  as  in  case  (1) ,  but  is  of  snaller 
magnitude  since  the  initial  estimate  is  based  on  a  better  approximation 
of  the  density  function.  Intuitively,  one  prefers  an  estimate  requir¬ 
ing  a  minimal  correction,  so  the  latter  method  was  used  in  the  com¬ 
parative  Monte  Carlo  tests  of  Section  3. 3.2.4. 
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b.  The  logarithmic  Interval  Histogram 


Data  taken  in  log-histogram  fo.m  can  be  considered  in  either 
of  two  domains,  as  discussed  in  Section  3.2.2.  In  the  £  domain,  the 
histogram  has  uniform  intervals  of  unit  length  such  that: 


"i  =  Prfi  <  5  <  i+1-l  (D.24) 

The  equivalent  probability  density  in  the  x  domain  is: 

-  Vh(i)  (b)1  <  x  <  (b)i+1  (D.25) 

where  the  interval  boundaries  cure: 

\  =  (b)1  i  85  0,1,2, .. .  ,M-1  (D. 26) 

and  the  interval  length  functions  are: 

h(i)  =  (b-1)  (b) 1  (D.27) 


for  all  b  >  1,  where  b  is  the  base  of  the  logarithmic  transforma¬ 
tion, 

e  =  i°V 

For  these  values  of  X^,  Equation  (D.12)  becomes: 

AE[x].  =  5i[(b1+1)3  -  (b1)3]  +  ~[(bi+1)2  -  (b1)2] 
or: 

AE[x]i  =  y-  (b3  -  1)  (b)3i  +  ^  (b2  -  1)  (b)2i 
The  values  of  and  can  be  shown  to  be: 
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(D.28a) 


m.  = 
1 


^i+i  -  Vi1 
(b  -  1)  (b  +  l)2(b)1 


and: 


B.  =  P.  -  m. 
111 


(b  +  1)  (b)1 


(D.28b) 


fran  Equations  (D.7)  and  (D.10)  respectively,  such  that: 


Ajtxl.  ,  P.  tWj  +mi(b)3i 


b3  -  1  (b  +  1)  (b2-  1) 
3  3 


or: 


P.  (b2  -  1)  (b)2i  m,  (b)3i 

AE^i  =  - 


111.  \UI 

+  -^2 -  (b  -  !)■ 


and  substituting  for  nt ,  we  obtain: 


P.  (b2-l)(b)21  ...  . .  2,..2i 

AE[x]  .  =  -i - - - k^b-D  (b) 

1  2  6(b+ 1) 


tPi+l  ‘  Pi-1]  (D'29) 


Equation  (D.29)  is  of  the  desired  form  in  that  it  consists  of  a  con¬ 
ventional  zero-order  estimate  term, 


Pi(b2  -  l)(b)2i 

5 


i+1 

P. 

l 


x  dx 


and  a  first-order  correction  term, 


(D.30a) 


1th  correction  =  7  JL>>>21  [p  .  p  , 

6(b  +  l)2  1+1  "-1 


(D.30b) 


A  more  convenient  form  of  the  correction  equation  involves  the  tk 
values  since  those  represent  the  actual  measured  data.  The  correction 
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equation  then  becanes: 


or: 


1th  correction  = 

6(b  +  ir 


^i+l 


Vi 


(b  -  1)  (b)1+1  (b-l)(b)1-1 


1th  correction  =  —  -  [it.  '-  b2ir.  .  ] 

f  /V  i  lit  l+l  1“ 1 


6(b  +  1) 


(D.31) 


Hie  end  conditions  for  the  log-histogram  do  not  simplify  as  readily  as 
they  did  for  the  uniform  interval  case,  but  are  particularly  important 
due  to  the  potential  moment  contribution  of  such  large  segments  at  a 
large  distance  frcm  the  origin.  (Note  that  the  end  condition  at  the 
origin  is  not  significant  due  to  its  relatively  small  effect  on  the 
moments.) 

At  the  upper  end  of  the  histogram,  we  will  assure  that  the 
nonexistent  interval  mirrors  the  value  of  the  last  actual  inter¬ 

val,  i.e.,  that  for  slope  considerations; 


P  =  P 
M  M-l 

Since  the  interval  is  b  times  as  long  as  the  M-lst 

the  value  must  then  be: 


(D.32) 

interval, 


"m  "  bVi 


CD. 33) 


such  that  the  slope  correction  to  the 


M-l' 


st 


interval  becomes: 


(b  -  1)  (b)^1 
6  (b  +  l)5 


O^M-i  "  b  V-2] 


(D.34) 


Hie  resulting  total  correction  is  therefore, 
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b  -  1 
6  (b 


{ M-2  ) 

“72  £  fVl  “  h\-l]  03)1  +  ^M-l  ‘  b2V2]  (b)M"(  <D-35> 

+  1)  {  i=i  ) 


with  the  uncorrected  estimate  being: 


.2  .M-l 

E[xJ._  =  h-y-l  £  Px  (b)21 


uc 


i=0 


or: 


E[x] 


b  +  1 


uc 


M-l 

i=0 


(D. 36) 


For  the  practiced  case  of  concern  to  us,  the  base  of  the  logarithmic 
transform  is  equal  to  two,  with  the  uncorrected  estimate  being: 


M-l 

£(x1uc  =  5  EV2’1  (d-37) 

i=0 

and  the  corresponding  correction  formula  being: 

( ) 

5?  E  '"i+l  -  4Vl> 12)1  +  t2Vl  -  4V2J  <2>|  (D-38> 

1=1  ’ 


The  second  moment  estimate  and  correction  can  be  found  by 

th 

evaluating  Equation  (D.  12)  for  n  =  2,  with  the  i  component  being: 

AE[x2].  =  4i[(b1+1)4  -  (b1)4]  +  ^[(bi+1)3  -  (b1)3] 
which  simplifies  to: 
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(D.39) 


m.  (b  -  1) 


AE[x2].  -  -i- 3 -  (b)41  +  -i 


B,  (b  -  1) 


(b) 


3i 


Substituting  the  value  of  Bi  from  Equation  (D.28b) ,  we  obtain: 

,4i 


AE[x*] .  =  P. 

l  l 


(b3  -  1)  (b)31 
- 3 - 


m.  (b) 


+  f  ft5 " 1)2(b2 "  x)3 


vrfiich  upon  substitution  for  mi  fran  Equation  (D.28a)  becomes: 

=  ,[<b3-l^(b)M|  +  bfr  l>(b2 -1)  W31»  .  p  , 

1  6(b  +  1)^  1+1  11 


Equation  (D.40)  consists  of  a  zero-order  estimate  component, 

ii+1 

(b)3i|  r 

T 


P. 

i 


(b3  -  1)  (b)3i 


I  r  2 

■J-A  x  Pi 

(b)1 


dx 


(D.41) 


and  a  first-order  correction  component: 

ith  oorrection  =  ~ 67f1  i)*  [Pi+i  "  Pi-i] 

If  we  substitute  for  the  values  as  in  Equation  (D.31)  we  obtain: 

,  2i 


i^1  correction  =  fo? 


6  (b  +  1)  [7ri+l  "  b  "i-l1 


(D.42aj) 


st 


except  for  the  N-l  term  which  is: 


(b  -  1)  f  )7N“2  2  . 

- 67E-+  j.y -  IbVl  "  b  V21 


(D.42b) 


The  zero-order  estimate  can  be  also  obtained  in  terms  of  by  sub¬ 
stitution  in  Equation  (D.41)  which  leads  to  an  uncorrected  estimate  of: 
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(D.43) 


£r  2,  b2  +  b  +  1  t1  -  a-*  2i 
E[x  ]  _  =  - g tt  (b) 


uc 


i=0 


For  the  limiting  case  of  b  approaching  unity,  the  histogram  becemes 
arbitrarily  close  to  the  continuous  case,  and  the  zero-order  second 
moment  is  seen  to  be  unbiased.  The  correction  term  goes  to  zero  in 
this  limiting  case,  as  can  be  seen  from  Equations  (D.42a,  b) .  For  the 
practical  case  of  concern,  b  =  2,  and  the  equations  become: 


A  2 

Zix] 


M-l 


uc 


-  1  J 

=  tE 


1^(2) 


2i 


and: 


i=0 


correction  = 


18  I  L, 

'  i=l 


[Vl 


-  47ri_1]  (4) 


2i 


(D.44) 


+  [2ttm_1  -  4ttm_2]  (4)K_1  J  (3.45) 

The  correction  formulas  for  the  first  and  second  moments  of  both 
uniform  and  log-histograms  were  utilized  in  the  Monte  Carlo  tests  as 
described  in  Section  3. 3. 2. 4,  and  on  the  average,  produced  significant 
improvements  in  the  moment  estimates.  In  same  cases  the  resulting 
estimates  were  overcorrected,  i.e. ,  produced  equally  large  errors  but 
of  the  opposite  sign,  and  in  isolated  cases,  produced  even  larger  errors 
than  the  "uncorrected"  estimates.  However,  more  typical  correction  re¬ 
sults  were  to  reduce  estimate  errors  as  shown  in  Table  3.3.2. 

The  effect  of  the  first-order  correction  for  the  exponential 
density  is  shown  in  Figure  D.2,  and  indicates  a  reasonable  slope  curve 
fit  over  each  segment  of  the  log-histogram.  However,  the  last  interval 
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D  100  300  NO  400  SDO 

x 

Fijuro  0.2.  An  Exponential  Density  Shown  with  the  Theoretical  Log-Histogram  Values 
and  the  First-Order  Slope  Approximation. 
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sho-s  an  effect  which  can  result  in  significant  errors,  and  is  shown 
on  a  magnified  scale  in  Figure  D.3.  The  first-order  slope,  in  this 
case ,  causes  the  approximating  function  to  br'cmr'  mxjntivo,  and  irr- 
tmluces  extra  positive  mass  at  the  lower  |>ortion  <>l  the  iulerv.il. 

Both  effects  tend  to  bias  the  moment  estimates  to  bo  lov,  with  an  in¬ 
creasingly  imoortant  effect  on  higher  mcronts.  Such  a  condition  can  be 
determined  fran  the  probability  heights,  i.e.,  it  will  occur  if: 


Figure  D.3.  An  Enlarged  Portion  of  the  First-Order  Approximation  of  Figure  D.2 
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n,  iWi>P. 

l  2  1 


(D.46) 


or  substituting  for  rru  and  reducing,  if: 

bfPi+l  "  Pi-1]  >  Pj.ffo  “  1)  (b  +  l)2] 

In  terms  of  the  ir^  probabilities,  the  test  beocrues : 


(D.47) 


[TTi+  -  b27T  ] 

-  >  (b  +  l)2  iri 


IK"-  1) 


(D.48) 


The  appropriate  action  to  take  when  the  test  fails  is  not  clear,  and 
probably  should  be  handled  on  an  individual  basis. 


330 


APPENDIX  E 


MOMENT  ESTIMATION  CAPABILITIES  OF 
UNIFORM  AND  LOG-HISTOGRAMS 

Monte  Carlo  tests  were  run  to  evaluate  the  nxxnent  estimating 
capabilities  of  relatively  coarse  histograms  with  both  uniform  and 
logarithmic  intervals .  These  tests  were  devised  to  check  for  ary  par¬ 
ticular  sensitivities  to  sanple  size ,  location  of  the  mean,  or  density 
shape. 

a.  Overview  of  the  Monte  Carlo  Tests 

Uniform  random  variables  over  the  range  of  0  to  1  were 
generated  by  a  conventional  psuedo- randan  number  generator,  and  were 
converted  into  the  density  of  interest  by  a  transformation  of  variables 
of  the  form, 

f  (x)  dx  =  g(y)  dy  (E.  1) 

where  f(x)  is  the  desired  density  function,  and 
(1  0  <  y  <  1 

g(y)  =  (E.2) 

( 0  otherwise 

The  random  variables  were  sorted  into  both  uniform  and  logarithmic 
histograms,  and  also  were  accumulated  as  a  son  and  a  sum  of  squares  to 
provide  a  knom  record  of  the  true  sanple  mean,  mean-square,  and  vari¬ 
ance.  Estimates  of  these  moments  were  then  made  from  the  histograms 
and  were  ccrpared  with  the  kncwn  sample  statistics.  The  test  included 
the  first-order  mcment  corrections  for  both  the  uniform  and 
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loo-histograms ,  as  well  as  evaluating  the  superposition  method  of 
Section  3. 3. 2. 2. 

b-  Selection  of  the  Parameters  for  the  Experiment 

A  variety  of  density  shapes  was  desired,  so  the  tests 
included  the  uniform,  Erlang (2),  exponential,  hyperexponential ,  and 
truncated  1/x  density  functions.  The  uniform  and  truncated  1/x 
densities  were  optimal  for  density  estimates  because  of  the  equal  area 
segments  for  the  uniform  and  log-histograms  respectively,  and  therefore 
were  included  in  the  moment  estimate  tests  as  well.  The  Erlang  (2), 
exponential,  and  hyperexponential  were  considered  to  be  mare  typical  of 
the  densities  that  might  be  encountered  and  represent  an  interesting 
spectrum  of  possible  density  shapes. 

The  range  of  the  random  variables  was  set  fairly  arbitrarily 
at  0  to  500,  and  for  base  two  logarithms,  resulted  in  10  increments  for 
the  lc  : -histogram.  Ten  intervals  of  size  50  were  then  selected  for  the 
uniform  histogram.  The  values  of  the  moments  for  the  uniform  and  1/x 
densities  were  defined  by  the  range,  and  so  were  not  free  variables. 
However,  the  Erlang (2),  exponential,  and  hyper  exponenti al  densities  had 
no  such  constraints,  but  their  parameters  were  arbitrarily  set  to  have 
approximately  the  same  mean  value  as  the  1/x  density.  The  means  are 
only  approximately  equal  due  to  the  truncation  effects,  i.e.,  no  values 
over  500  are  allowed.  This  selection  of  intervals  and  mean  values 
should  be  a  highly  unfavorable  test  for  the  log-histogram  since  seven 
its  ten  intervals  fall  well  below  the  mean  value. 

The  density  functions  being  considered  are  shown  in  Table  E.l 
which  lists  their  parameter  values  and  the  resulting  mean,  variance. 
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and  coefficient  of  variation,  where  these  latter  values  include  the 
truncation  effect. 


TABLE  E.l 

THEORETICAL  VALUES  OF  THE  MOMENTS  OF  IKE 
TRUNCATED  DENSITY  FUNCTIONS  BEING  CONSIDERED 


Density 

Range 

Parameters 

Mean 

Variance 

Coefficient 
of  Variation 

Uniform 

0-500 

250.0 

20,830 

0.58 

Erlang (2) 

0-500 

80 

80.0 

3,170 

0.70 

Eixponential 

0-500 

80 

79.0 

5,960 

0.98 

Hyperexponential 

0-500 

40  and  120 

76.0 

8,820 

1.23 

1/x  density 

1-500 

_L  80,3 

13,670 

1.46 

c.  Generation  of  the  Desired  Densities 

The  random  numbers  that  were  obtained  from  the  random  number 
generator  were  uniformly  distributed  over  the  range  of  zero  to  one. 

If  we  call  this  uniform  density  g(y)  where: 

(1  0  <  y  <  1 

g  (y)  =  I 

( 0  otherwise 

and  desire  a  transformed  density,  f  (x) ,  then  v/e  can  relate  the  two 
densities  by: 

f (x)  dx  =  g (y)  dy 

which  for  g(y)  =1  becomes: 
x 

y  =  f  f(x)  dx  =  F(x)  (E.3) 

0 
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We  want  the  inverse  function,  i.e.,  the  relationship  of  x  to  same 
function  of  y,  and  can  most  easily  obtain  this  by  a  slight  change  cf 
v  riable, 

y'  =  l-  y  =  l-  F(x)  (E.4) 

where  y '  will  also  be  uniformly  distributed  over  the  unit  interval. 
For  exanple,  if  f(x)  is  the  exponential  density, 

f(x)  =  y  e~MX  0  <  x  (E.5) 

then, 

F(x)  -  1  -  e“VX 

and 

y'  »  1  -  F (x)  =  e_MX 

so  that: 

ln(y')  (E.6) 

A  similar  transformation  can  be  found  for  the  truncated  1/x  density, 
f(x)  =  1/x  In  M  (E.7) 

with: 

y'  =  1  -  F (x)  =  In  x/ln  M 
or: 

x  =  exp(y'  In  M)  (E.8) 

The  transformations  for  the  Erlang  and  hyperexponential  densities  can 
readily  be  found  from  exponential  carponents;  the  Erlang  (2)  being  the 
sum  of  two  exponential  random  variables ,  and  the  hyperexponential  being 
the  random  selection  between  tvro  equally  likely  exponential  random 
variables  with  different  means. 
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d.  Moment  Estimates  Using  the  Superposition  Method 

The  technique  described  in  Section  3. 3. 2. 2,  in  which  the  log- 
histogram  corponents  were  transformed  into  1/x  density  segments,  was 
evaluated  by  a  series  of  Monte  Carlo  tests  with  uniform.  Erlang  (2) , 
exponential,  hyperexponential ,  and  truncated  1/x  density  functions. 
The  method  produced  excellent  results  for  the  1/x  density,  as  would 
be  expected,  and  somewhat  surprisingly,  also  gave  good  results  for  the 
uniform  density.  However ,  for  the  Erlang  (2),  exponential,  and  hyper¬ 
exponential  cases,  the  estimates  of  the  mean  were  in  error  by  about  5% 
to  6%,  and  the  variance  estimates  were  about  30%  too  large. 

The  reason  for  both  the  good  estimates  of  the  uniform  density 
case,  and  the  poor  estimates  of  the  "long  tail"  densities,  is  the  fact 
that  the  1/x  density  is  reasonably  flat  at  the  larger  values  of  x. 
Ibis  upper  region  provides  a  predominant  contribution  to  the  moment 
calculation,  and  must  be  approximated  quite  closely  if  good  estimates 
are  to  be  obtained.  Since  the  1/x  density  approximation  has  a  sig¬ 
nificant  weakness  in  this  area,  it  was  not  pursued  any  further. 

e.  Moment  Estimates  Using  the  First-Order  Correction  Bqua tiers 

The  five  density  functions  shown  in  Table  E.l  were  utilized 
as  test  cases  to  evaluate  and  oerpare  the  moment  estimation  capabilities 
of  the  first-order  correction  equations  for  both  uniform  and  log-histo¬ 
grams.  The  resulting  estimates  of  the  mean  values  are  shown  in  Table 
E.2  with  the  variance  estimates  being  shown  in  Table  E.3.  The  sur¬ 
prising  result  is  that  the  estimates  from  the  log-histograms  were  often 
actually  better  than  those  from  the  uniform  histogram,  even  though  the 
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TABLE  E.2 

A  COMPARISON  OF  ESTIMATES  OF  THE  MEAN  FROM  UNIFORM  AND  LOG-HISTOGRAMS 


Density  Lot 


Function 


Uniform 


Erlang  (2) 


Total 


Hyper¬ 

exponential 


Total 


Truncated 
1/x  density 


Total 


Estimates  of  the  Mean 

Sample 

Mean 

Uniform  Histogram 

Log-Histogram 

Estimate 

%  Error 

Estimate 

%  Error 

242 

242 

EM 

244 

+0.7% 

250 

250 

247 

-0.9% 

243 

244 

K| 

250 

+2.8% 

253 

254 

899 

256 

+1.2% 

247 

248 

+0.3% 

249 

+1.0% 

84.5 

84.5 

+0.0% 

84.7 

+0.2% 

90.1 

89.6 

-0.6 

87.0 

-3.5% 

75.0 

72.3 

-3.7% 

74.6 

-0.6% 

78.4 

77.4 

-1.2% 

79.3 

+1.1% 

82.0 

80.9 

-1.4% 

81.4 

-0.7% 

81.8 

83.7 

81.8 

+0.0% 

82.4 

85.1 

81.9 

-0.7% 

80.5 

81.1 

81.2 

+0.9% 

78.1 

80.2 

1 

76.0 

-2.8% 

80.7 

82.5 

i-  - 

+2.3%  | 

80.2 

-0.6% 

84.9 

86.9 

+2.4% 

84.1 

-0.9% 

73.3 

75.4 

+2.8% 

74.3 

+1.3% 

81.7 

82.6 

+1.0% 

82.8 

+1.3% 

83.4 

86.7 

+3.9% 

83.0 

-0.6% 

80.8 

82.9 

+2.5% 

81.0 

+0.3% 

80.2 

87.9 

+9.6% 

81.7 

+1.9% 

91.2 

98.8 

+3.3% 

93.8 

+2.8% 

73.5 

80.0 

+8.9% 

71.5 

-2.8% 

78.6 

85.8 

+9.0% 

82.0 

+4.3% 

80.9 

88.1 

+9.0% 

82.2 

+1. 6% 

TABLE  E.3 


A  COMPARISON  OF  ESTIMATES  OF  THE 
VARIANCE  FROM  UNIFORM  AND  LOG-HISTOGRAMS 


r 

Estimates  of  the  Variance 

:  i 

Density 

Function 

Lot 

Sarrple 

Variance 

Uniform  Histogram 

Log-Histogram  ! 

No. 

Estimate 

%  Error 

1  Estimate 

%  Error 

1 

21,800 

23.300 

+  6.6% 

22,100 

+  1.2% 

Uniform 

2 

23,700 

25,100 

+  5.8% 

22,800 

-  3.8% 

3 

19,600 

-s.  ■ 

+  6.6% 

21,000 

+  7.1% 

4 

20,200 

1  ■ 

+  0.1% 

21,600 

+  6.8% 

Total 

21,300 

22,800 

+  7.0% 

+  2.8% 

WSM 

3,470 

4,110 

+18.6% 

2,200 

-36.5% 

1 

MrTi.C' 

4,810 

+18.9% 

2,390 

-41.0% 

Erlang  (2) 

3,020 

+22.2% 

1,080 

-56.3% 

1 

3,810 

+21.5% 

2,940 

-  6.1% 

L 

Total 

3,940 

+20.3% 

2,150 

-35.0% 

1 

1 

5,650 

6,110 

+  8.3% 

5,300 

-  6.1% 

2 

7,450 

+  7.1% 

6,820 

-  1.9% 

Exponential 

3 

5,610 

6,200 

+10.4% 

6,010 

+  7.1% 

1 

4 

6,160 

6,530 

+  6.1% 

4,890 

-20.6%  i 

Total 

6,090 

6,570 

+  8.0% 

5,760 

-  5.4% 

1 

7,100 

7,510 

+  5.7% 

6,250 

-12.0% 

Hyper- 

2 

7,090 

7,270 

+  2.6% 

7,260 

+  2.3% 

exponential 

3 

7,540 

7,880 

+  4.5% 

7,410 

-  1.8% 

4 

7,510 

7,860 

+  4.6% 

6,940 

-  7.6% 

Total 

7,310 

J 

7,630 

+  4.3% 

6,960 

-  4.8%  | 
_ J 

Urn 

14,500 

13,900 

-  4.7% 

| 

15,000 

+  2.8%  1 

Truncated 

H 

17,200 

-  4.2% 

1 

17,600 

+  2.3% 

1/x  density 

1 

12,500  | 

12,000 

-  4.1% 

12,000 

-  3.8%  1 

mm 

12,300 

IB 

-  3.2% 

■EbB 

+15.2% 

Total 

14,100 

13,600 

-  4.0% 

14,700 

+4.1%  j 
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latter  has  much  better  resolution  at  the  higher  data  values,  and 
consequently  should  yield  better  estimates. 


Some  insight  into  this  apparent  discrepency  can  be  gained 
from  Table  E.4  which  compares  the  corrected  and  uncorrected  estimates 


TABLE  E.4 

THE  EFFECT  OF  THE  FIRST-ORDER  MOMENT 
CORRECTIONS  FOR  AN  EXPONENTIAL  DENSITY 


Mean 

Mean  Square 

Variance 

Uniform  intervals 

m 

+  7.9% 

+  6.3% 

+  9.2% 

+  8.0% 

•  uncorrected 

•  corrected 

Log  intervals 

+6.9% 

-0.6% 

+21.4% 

-  3.2% 

+29.3% 

-  5.4% 

•  uncorrected 

•  corrected 

for  the  two  types  of  histograms  with  an  exponential  density  function. 
The  uncorrected  estimates  of  the  first  and  second  moments  have  rela¬ 
tively  large  errors  for  the  log-histogram  procedure,  but  the  correction 
can  be  seen  to  be  quite  effective  in  decreasing  the  error;  and  in  fact 
tends  to  somewhat  overcorrect  the  values, producing  an  error  of  the 
opposite  sign.  In  contrast,  the  uniform  histogram  corrections  are  not 
nearly  large  enough  and  Tables  E.2  and  E.3  show  that  this  was  typically 
the  case.  The  log-histogram  corrections  were  far  more  effective  and 
produced  overall  estimates  which  appeared  to  be  reasonably  unbiased. 

The  same  basic  correction  technique  was  used  for  both  types  of  histo¬ 
grams,  as  described  in  Appendix  D,  but  the  data  indicates  that  seme 


338 


other  weighting  of  the  adjacent  slopes  would  probably  give  better 
results  for  the  uniform  histogram,  since  they  should  inherently  give 
better  mcnent  estimates  than  the  log-histograms.  This  subject  was  not 
pursued,  since  for  our  purposes  the  log-histograms  were  shewn  to  be 
preferable  for  density  estimates,  and  when  corrected,  produce  good 
estimates  of  at  least  the  first  and  second  moments. 

The  five  density  functions  were  selected  to  have  a  variety  of 
density  shapes ,  and  indeed  some  shape-dependent  errors  were  seen  in  the 
test  data.  For  exanple,  the  Erlang  (2)  density  test  was  found  to  have 
very  large  errors  in  the  variance  estimates.  This  effect  was  due  to 
two  factors;  (a)  the  second  moment  about  the  origin  is  somewhat  over- 
corrected,  and  (b)  the  variance  is  the  difference  between  two  estimates, 

a2  =  E[x2]  -  (Elx]}2  (E.7) 

and  since  the  variance  of  the  Erlang  density  is  relatively  small,  this 
becomes  a  large  percentage  error.  For  exanple,  in  the  first  sample, 
the  errors  in  the  mean  and  meansquare  values  were  0.2%  and  -11.6% 
respectively,  but  the  resulting  error  in  the  variance  was  -36.5%.  For¬ 
tunately,  the  Erlang  density  shape  is  not  expected  in  any  of  the  message 
size  data,  so  the  log-histogram  data  reduction  for  the  network  measure¬ 
ments  should  not  involve  this  kind  of  error. 

f .  A  Comparison  of  the  Estimates  for  Different  Mean  Values 

In  many  measurement  situations,  one  knows  the  range  of  allow¬ 
able  values  but  does  not  have  much  insight  into  the  expected  value  of 
the  variable.  We  will  now  compare  the  uniform  and  log-histogram  for 
such  a  case,  with  the  value  constrained  to  be  within  the  range  of  0  to 
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500,  but  with  various  values  of  the  mean.  The  resulting  errors  are 
plotted  in  Figure  E.l  as  a  function  of  the  sarrple  mean  (for  an  exponen¬ 
tial  density) .  The  log-histogram  estimates  have  errors  in  the  estimate 
of  the  mean  value  which  are  reasonably  independent  of  the  actual  mean 
value,  while  the  uniform  histogram  estimates  are  a  strong  function  of 
the  location  of  the  mean.  While  this  exarrple  does  not  necessarily  in- 
ciated  a  general  relationship,  it  does  seen  to  be  reasonable  to  expect 
many  distributions  to  have  such  a  characteristic  because  of  the  nature 
of  the  errors  involved.  That  is,  the  uniform  histogram  will  be  expected 
to  have  a  reasonably  constant  absolute  error  in  its  estimates,  which 
will  be  a  decreasing  percentage  error  as  the  nominal  mean  value  is  in¬ 
creased.  In  contrast,  the  log-histogram  has  an  absolute  error  which  is 
approximately  proportional  to  the  nominal  value ,  and  hence  produces  a 
fairly  constant  percentage  error. 

The  second  moments  are  seen  to  have  a  larger  percentage  error 
in  both  cases,  although  the  log -his  bogran  has  scmewhat  less  error  than 
the  uniform  histogram.  In  contrast,  the  variance  estimates  of  the  uni¬ 
form  histogram  are  reasonably  constant  over  the  entire  range,  and  are 
often  less  than  those  of  the  log-histogram.  This  is  due  to  the  fact 
that,  for  the  exponential  density,  the  error  in  the  variance  is  propor¬ 
tional  to  the  difference  between  the  percentage  errors  of  the  first  and 
second  mcments, 


er(c2) 


S  2 


er(x  ) 


“7 

x 


er(x) 

x 


(E.8) 
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PERCENTAGE  ERROR  IN  THE  ESTIMATE 


LOCATION  OF  THE  MEAN  VALUE 


Figure  E.l.  A  Comparison  of  the  Errors  in  the  Moment  Estimates 
from  Uniform  and  Log-Histograms. 
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aid  this  difference  is  fairly  constant  for  the  uniform  histogram  case. 
This  relationship  can  be  shewn  by  considering  the  error  of  the 
estimates: 


er(x)  =  Efx]  -  Efx] 
er(?)  =  Efx2]  =  Efx2] 
er(o2)  =  a2  -  a2 


The  estimated  variance  is: 


(E.9a) 
(E.  9b) 
(E.9c) 


aO  ^  9  /s  9 

o  =  Efx^]  -  (Efx] ) ^ 


(E.10) 


and  the  actual  variance  is: 

a2  =  Efx2]  -  (Efx])2  (E.  11) 

If  we  subtract  Equation  (E.ll)  from  Equation  (E.10) ,  we  obtain  the 
error  in  the  variance, 

02  -  a2  -  (Efx2]  -  Efx2]}  -  {(Efx])2  -  (Efx])2} 
which  can  be  written  as: 


er(o2)  =  er(j?)  -  {Efx]  +  Efx]}  •  er(x) 


(E.12) 


For  small  errors ,  the  sun  term  will  be  approximately  equal  to  twice 
the  mean  value,  such  that: 


er  (a2)  =  er(?)  -  2  x  er(x) 


(E.13) 


This  expression  simplifies  for  the  case  of  the  exponential  density  with, 

(E.14) 


~~7  2  —  2 

x  =  2o  -  2(xj 
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resulting  in: 


er(o^)  er(x^)  _  er  (x) 
which  yields  Equation  (E.8). 
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GLOSSARY 


Accumulated  statistics:  The  measurement  routine  which  gathers  counts, 
suns,  and  histograms  of  activity  in  an  IMP  over  a  time  period 
(presently  12.8  seconds). 

ACK,  acknowledgement :  An  IMP- to- IMP  transmission  from  JMk  to  IMP^ 

that  verifies  that  a  packet  was  received  and  accepted  by  2Mk 
from  IMPj  along  the  store-and- forward  path. 

ARPA:  The  Advanced  Research  Projects  Agency,  the  agency  of  the  Depart¬ 
ment  of  Defense  which  sponsored  the  network  development. 

Artifact:  Any  measurement  effect  which  results  in  a  difference  between 
the  observed  system  variables  and  the  actual  system  variables 
(as  they  would  exist  if  the  measurements  had  not  been  made) . 

Artificial  traffic:  Traffic  that  is  generated  solely  for  purposes  of 
network  loading  and  tests. 

BBN:  Bolt  Beranek  and  Newman,  Inc.,  the  network  contractor  and  a  node 
in  the  network. 

Blocking:  The  condition  that  occurs  when  an  IMP  cannot  acoept  trans¬ 
missions  from  its  neighbors  due  to  the  lack  of  buffers. 

Buffer:  Storage  space  for  a  full  packet. 

Fake-  -iOST:  An  IMP  routine  which  performs  one  or  more  of  the  HDST-like 
functions,  e.g.,  the  source  or  destination  of  messages  such 
as  the  statistics  data,  the  IMP  console  Teletype,  and  the 

Preceding  page  blank 
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discard  facility. 

FCFS:  Firs t-Ccme-First- Served  queue  discipline. 

First-order  moment  correction:  A  correction  technique  for  nanent  esti¬ 
mates  from  his  tog  ran  data  using  a  linear  slope  approxirration 
based  on  neighboring  interval  heights. 

GHOST:  Same  as  a  Fake -HOST. 

Header:  A  64-bit  record  preceding  each  packet,  which  contains  the 
source,  the  destination,  the  message  number,  etc. 

HOP:  One  IMP-to-IM0  path  or  leg.  The  HOP  #  is  the  number  of  such  legs 
between  two  nodes. 

HOST:  One  of  the  independent  computing  centers  of  the  network. 

Hello:  A  periodic  transmission  between  adjacent  IMP's  to  pass  routing 
information  and  to  test  the  operational  status  of  the  lines. 

HOST-to-HOST  protocol:  The  prescribed  manner  in  which  HOST  computers 
ccrttm  uni  cate . 

IHY:  I  Heard  You;  the  response  to  a  Hello  message. 

IMP:  Interface  Message  Processor;  the  store- and-forward  message 
switches  for  the  network. 

Inter-packet  gap:  The  time  interval  that  occurs  between  otherwise  con¬ 
tiguous  packet  flew  due  to  other  traffic  on  the  communication 
channels. 

Leader:  The  32-bits  of  information  which  precede  each  message  and 
define  its  destination,  message  number,  etc. 
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Link:  A  logical  (software)  connection  between  the  IMP's  at  the  source 
and  destination  nodes  of  a  message  transmission.  The  link 
becomes  "blocked",  (i.e.,  cannot  be  used  again)  until  a  RFNM 
is  returned. 

L  L:  Lincoln  Laboratories 

Log-histogram:  A  histogram  in  which  the  intervals  are  scaled  in  a 

logarithmic  (or  exponential)  manner.  In  the  case  of  binary 
log-histograms,  each  interval  is  twice  the  size  of  the 
preceding  inberval. 

Marking:  A  bit-string  00... 01  preceding  the  first  bit  of  a  message  (for 
transmissions  between  HOST  ccnputers  having  differing  word 
lengths) . 

MIT:  Massachusetts  Institute  of  Technology 

Message:  A  logical  record  of  up  to  8095  bits  which  is  used  for  HOST-to- 
IJOST  transmissions. 

Modem:  A  modulator-demodulator  used  to  convert  between  digital  data 
and  the  frequency-modulated  telephone  carrier  signal. 

Moment  corrections:  The  addition  of  term(s)  to  compensate  for  the 

systematic  bias  in  moment  estimates  from  group  xi  data  such  as 
histograms . 

.  -3 

msec:  millisecond,  i.e.,  10  seconds. 

Usee:  microsecond,  i.e.,  10  ^  seconds. 

Net,  network.  The  communications  facilities  (including  IMP's)  which 
interconnect  the  HOST  computers .  Sometimes  used  in  a  larger 
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scope  including  the  HOST  ocnputers  as  well. 

NMO:  The  Network  Measurement  Center  (at  UCLA) . 

Node:  One  of  the  sites  involved  in  the  network,  i.e.,  typically 
referring  to  both  the  IMP  and  HOST(s). 

Packet:  A  message  segment  of  up  to  1008  bits  of  information,  (the 
standard  IMP-to-IMP  transnission  block  size) . 

Padding:  A  bit-string  10... 00  following  the  last  bit  of  a  message  to 
fill  out  the  remaining  portion  of  the  IMP  (or  destination 
HOST)  word  length. 

Poisson  arrivals:  Independent  random  arrivals  to  a  queueing  system 
(with  an  exponential  density  of  inter  arrival  times) . 

Propagation  delay:  The  transmission  delay  due  to  the  finite  propaga¬ 
tion  time  of  electrical  energy  along  a  transmission  line 
(about  10  usee,  per  mile) . 

Protocol :  The  established  procedures  and  standards  for  network 
transmissions. 

Pseudo-messages:  Fixed  length  artificial  traffic  generated  by  the 
IMP's. 

RAND:  The  RAND  Corporation. 

Reassembly:  The  process  of  collecting  packets  and  placing  than  in  the 
proper  sequence  to  recreate  the  segmented  message. 

RINM:  Request  For  Next  Message;  the  response  which  is  returned  to  the 
message  source  to  allow  further  transmission  along  a  given 
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link.  (A  congestion  and  flow  control  mechanism. ) 


RFNM  driven  traffic:  Traffic  flew  that  occurs  as  rapdily  as  the  RFNM 
mechanism  will  allow. 

Routing:  The  selection  of  each  s tore- ard- forward  "hop"  along  the 
source- to-des ti nation  path. 

Routing  table:  A  table  listing  the  estimated  best  known  ccrtmunication 
channels  for  transmission  to  the  various  destinations. 

Routing  update:  Hie  process  of  exchanging  routing  information  between 
the  nodes  in  a  network. 

SDC:  System  Development  Corporation. 

Signa  7:  Hie  computer  system  at  the  UCLA  Network  Measurement  Center. 

Segment:  A  portion  of  a  message;  often  synonymous  with  packet. 

Segmented  messages:  The  division  of  messages  into  fixed  size  blocks 
(called  packets) . 

SENT  queue:  A  linked  list  of  packets  which  have  been  transmitted,  but 
not  yet  acknowledged. 

Snap-shots:  The  measurement  routine  which  records  the  state  of  the  IMP 
queues  and  routing  tables  at  periodic  intervals  (presently 
every  0.8  sec.) . 

SRI:  Stanford  Research  Institute. 

Store-and-f orward :  The  receipt  and  subsequent  transmission  of  a  packet 
by  an  intermediate  node  in  a  network  source-destination  path. 
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'fruru-vut:  The  effective  rate  at  which  data  can  be  transmitted  through 
the  network,  i.e. ,  between  a  given  source  and  destination. 

Praoes :  Measurement  data  taken  at  an  IMP  recording  the  time  at  which 
a  given  packet  was  received  and  "processed" . 

UCLA:  University  of  California  at  Los  Angeles. 

UCSB:  University  of  California  at  Santa  Barbara. 

Unfinished  work  diagram:  A  diagram  showing  the  amount  of  the  service 
backlog"  which  iu  present  in  the  systan  as  a  function  of 
time. 

Uniform  histogram:  A  conventional  histogram  in  which  all  of  the 
intervals  are  of  equal  size. 

Utah:  The  University  of  Utah. 
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